Are some ways of measuring scientific quality better than others? Sune Lehmann, Andrew D. Jackson and Benny E. Lautrup analyse the reliability of commonly used methods for comparing citation records. Citation analysis can loom large in a scientist's career. In this issue Sune Lehmann, Andrew Jackson and Benny Lautrup compare commonly used measures of author quality. The mean number of citations per paper emerges as a better indicator than the more complex Hirsch index; a third method, the number of papers published per year, measures industry rather than ability. Careful citation analyses are useful, but Lehmann et al. caution that institutions often place too much faith in decisions reached by algorithm, use poor methodology or rely on inferior data sets.