ABSTRACTMany seem to believe that researcher‐made tests are unnecessary, if not inappropriate, for evaluating reading comprehension interventions. We suggest that this view reflects a zeitgeist in which researcher‐made (proximal) tests that align with the researchers’ interventions are closely scrutinized and often devalued, whereas commercially developed (distal) tests, typically unaligned with the researchers’ interventions, escape such examination and judgment. We take issue with the zeitgeist. We object to what appears as an unthoughtful rejection of proximal tests and acceptance of the distal ones. We do so first by discussing the multidimensionality of reading comprehension; then, we present evidence that commercial tests explore few of the construct’s many dimensions, differ among themselves in the dimensions they address, and often have little to do with the aims and substance of researchers’ comprehension interventions. We argue that these facts are reason enough to reconsider commercial tests as the gold standard in program evaluation. Finally, we make a case for the supplemental use of researcher‐made tests and offer a framework to help researchers develop tests of reading comprehension that align more and less with their intervention programs.