Abstract Although the benefits of extensive reading are well known, very little research has investigated the validity of assessment tools to measure general comprehension of graded readers. To address this, quizzes were authored for 42 level 2 Penguin graded readers and administered to 166 students over a semester. Item facility for high-scorers and low-scorers was calculated for the 168 question items from the 42 graded readers, and the resulting item discrimination (ID) score was used to categorize and group quiz items with shared structural and content-based features. The results showed that certain question types tend to be more effective for measuring comprehension of graded readers than others.