Multiple choice questions (MCQs) are widely used as an efficient means to grade large batches of students. With technology enabling extremely large classes (MOOCs), the use of MCQs has increased rapidly, leading to an increased scrutiny of their pedagogical utility. In this paper, we present a variant of MCQs that requires students to justify their answer by choosing one or more supporting statements from an instructor-defined list. Thus we retain the ability to automate the grading process, while addressing some (but not all) of the known weaknesses of such assessment. To help the educational research community evaluate the pedagogical utility of this approach, we have created an open-source plug in for creating and evaluating such questions on a widely used e-learning platform (Moodle).