Abstract Rhabdoviruses with rich species lead a variety of high lethality and rapid transmission diseases to plants and animals around the globe. Vaccination is one of the most effective approaches to prevent and control virus disease. However, the key antigenic epitopes of glycoprotein being used for vaccine development are unclear. In this study, fish-derived Abs are employed for a Micropterus salmoides rhabdovirus (MSRV) vaccine design by phage display and bioinformatics analysis. We constructed an anti-MSRV phage Ab library to screen Abs for glycoprotein segment 2 (G2) (G129-266). Four M13-phage-displayed Abs (Ab-5, Ab-7, Ab-8 and Ab-30) exhibited strong specificity to target Ag, and Ab-7 had the highest affinity with MSRV. Ab-7 (300 μg/ml) significantly increased grass carp ovary cell viability to 83.40% and significantly decreased the titer of MSRV. Molecular docking results showed that the key region of Ag–Ab interaction was located in 10ESQEFTTLTSH20 of G2. G2Ser11 and G2Gln12 were replaced with alanine, respectively, and molecular docking results showed that the Ag–Ab was nonbinding (ΔG > 0). Then, the peptide vaccine KLH-G210–20 was immunized to M. salmoides via i.p. injection. ELISA result showed that the serum Ab potency level increased significantly (p < 0.01). More importantly, the challenge test demonstrated that the peptide vaccine elicited robust protection against MSRV invasion, and the relative percentage survival reached 62.07%. Overall, this study proposed an approach for screening key epitope by combining phage display technology and bioinformatics tools to provide a reliable theoretical reference for the prevention and control of viral diseases.