The macrodomain of nsP3 (nsP3MD) is highly conserved among the alphaviruses and ADP-ribosylhydrolase activity of Chikungunya Virus (CHIKV) nsP3MD is critical for CHIKV viral replication and virulence. No small molecule drugs targeting CHIKV nsP3 have been identified to date. Here we report small fragments that bind to nsP3MD which were discovered by virtually screening a fragment library and X-ray crystallography. These identified fragments share a similar scaffold, 2-pyrimidone-4-carboxylic acid, and are specifically bound to the ADP-ribose binding site of nsP3MD. Among the fragments, 2-oxo-5,6-benzopyrimidine-4-carboxylic acid showed anti-CHIKV activity with an IC50 of 23 μM. Our fragment-based drug discovery approach provides valuable information to further develop a specific and potent nsP3 inhibitor of CHIKV viral replication based on the 2-pyrimidone-4-carboxylic acid scaffold. In silico studies suggest this pyrimidone scaffold could also bind to the macrodomains of other alphaviruses and coronaviruses and thus, have potential pan-antiviral activity.