B-cell antibodies are vital proteins offering robust protection for the human body from pathogens. The development of general protein and antibody-specific pre-trained language models facilitates antibody prediction tasks. However, few studies comprehensively explore the representation capability of distinct pre-trained language models on different antibody problems. Previously, no benchmark available largely hindered the survey to answer these questions. We provide an AnTibody Understanding Evaluation (ATUE) benchmark to facilitate the investigation. We comprehensively evaluate the performance of protein pre-trained language models by empirical study along with conclusions and new insights. The related manuscript can be found in biorxiv with title "On Pre-trained language models for antibody".