Complement or Contamination: A Study of the Validity of Multiple-Choice Items when Assessing Reasoning Skills in Physics

Article English OPEN
Jönsson, Anders ; Rosenlund, David ; Alvén, Fredrik (2017)
  • Publisher: Frontiers Media S.A.
  • Journal: Frontiers in Education, volume 2 (issn: 2504-284X)
  • Related identifiers: doi: 10.3389/feduc.2017.00048/full, doi: 10.3389/feduc.2017.00048
  • Subject: assessment | Education (General) | Didactics | Education | L7-991 | Argumentation skills; assessment; multiple-choice items; national testing; socio-scientific issues | socio-scientific issues | national testing | Didaktik | argumentation skills | multiple-choice items
    acm: ComputingMilieux_COMPUTERSANDEDUCATION

The purpose of this study is to investigate the validity of using multiple-choice (MC) items as a complement to constructed-response (CR) items when making decisions about student performance on reasoning tasks. CR items from a national test in physics have been reformulated into MC items and students’ reasoning skills have been analyzed in two substudies. In the first study, 12 students answered the MC items and were asked to explain their answers orally. In the second study, 102 students from five randomly chosen schools answered the same items. Their answers were scored, and the frequency of correct answers was calculated for each of the items. The scores were then compared to a sample of student performance on the original CR items from the national test. Findings suggest that results from MC items might be misleading when making decisions about student performance on reasoning tasks, since students use other skills when answering the items than is intended. Results from MC items may also contribute to an overestimation of students’ knowledge in science.
  • References (26)
    26 references, page 1 of 3

    Becker W. E. Johnston C. (1999). The relationship between multiple choice and essay response questions in assessing economics understanding. Econ. Rec. 75, 348–357.10.1111/j.1475-4932.1999.tb02571.x

    Bennett R. E. (1993). “On the meaning of constructed response,” in Construction Versus Choice in Cognitive Measurement. Issues in Constructed Response, Performance Testing, and Portfolio Assessment, eds Bennett R. E. Ward W. C. (Hillsdale, NJ: Lawrence Erlbaum Associates), 1–27.

    Bible L. Simkin M. Kuechler W. (2007). How well do multiple-choice tests evaluate students’ understanding of accounting? Account. Educ. 17, 55–68.10.1080/09639280802009249

    Bond T. G. Fox C. M. (2007). Applying the Rasch Model. Fundamental Measurement in the Human Sciences. New York, London: Routledge.

    Christenson N. Chang Rundgren S.-N. (2015). A framework for teachers’ assessment of socio-scientific argumentation: an example using the GMO issue. J. Biol. Educ. 49, 204–212.10.1080/00219266.2014.923486

    Dufresne R. J. William L. J. William G. J. (2002). Making sense of students’ answers to multiple-choice questions. Phys. Teach. 40, 174–180.10.1119/1.1466554

    Dunbar S. B. Koretz D. M. Hoover H. D. (1991). Quality control in the development and use of performance assessments. Appl. Meas. Educ. 4, 289–303.10.1207/s15324818ame0404_3

    Gustafsson J.-E. Cliffordson C. Erickson G. (2014). Likvärdig kunskapsbedömning i och av den svenska skolan – problem och möjligheter [Equity in Assessment in and of the Swedish School – Problems and Possibilities]. Stockholm: SNS Förlag.

    Hamilton L. S. Nussbaum E. M. Snow R. E. (1997). Interview procedures for validating science assessments. Appl. Meas. Educ. 10, 181–200.10.1207/s15324818ame1002_5

    Hogan T. P. (1981). Relationship between Free-Response and Choice-Type Tests of Achievement: A Review of the Literature. Washington, DC: National Institute of Education.

  • Metrics
    No metrics available
Share - Bookmark