There has been much philosophical interest regarding the 'hierarchy of evidence' used to determine which study designs are of most value for reporting on questions of effectiveness, prognosis, and so on. There has been much less philosophical interest in the choice of outcome measures with which the results of, say, an RCT or a cohort study are presented. In this paper, we examine the FDA's recently published guidelines for assessing the psychometric adequacy of patient-reported outcome measures. We focus on their recommendations for demonstrating content validity and also for how researchers should weigh up the sum of psychometric evidence when choosing these measures. We argue that questions regarding judgment and understanding meaning of these measures should play a more central role in determining their adequacy.