Do Students Behave Rationally in Multiple Choice Tests? Evidence from a Field Experiment

Article OPEN
María Paz Espinosa ; Javier Gardeazabal (2013)
  • Journal: Journal of Economics and Management, volume 9, issue 2 July, pages 107-135
  • Subject: scoring rules, risk aversion, field experiment
    • jel: jel:D03 | jel:C93 | jel:D81

A disadvantage of multiple choice tests is that students have incentives to guess. To discourage guessing, it is common to use scoring rules that either penalize wrong answers or reward omissions. In psychometrics, penalty and reward scoring rules are considered equivalent. However, experimental evidence indicates that students behave differently under penalty or reward scoring rules. These differences have been attributed to the different framing (penalty versus reward). In this paper, we model students’ behavior in multiple choice tests as a choice among lotteries. We show that strategic equivalence among penalty and reward scoring rules holds only under risk neutrality. Therefore, risk aversion could be an alternative explanation to the previously found differences in students’ behavior when confronted with penalty and reward scoring rules. We suggest the use of a modified penalty scoring rule which is equivalent to the reward rule for whatever risk attitudes students might have. To disentangle the effect of framing and risk aversion on students’behavior we design a field experiment with three treatments, each one with a different scoring rule. Two of these scoring rules are equivalent but have different framing, while the third is not equivalent but has the same framing as one of the other two. The experimental results indicate that differences in students’ behavior are due to risk aversion and not due to different framing.
  • References (25)
    25 references, page 1 of 3

    Becker, W. E. and C. Johnston, (1999), “The Relationship between Multiple Choice and Essay Response Questions in Assessing Economics Understanding,” The Economic Record, 75, 348-357.

    Bereby-Meyer, Y., J. Meyer, and O. M. Flascher, (2002), “Prospect Theory Analysis of Guessing in Multiple Choice Tests,” Journal of Behavioral Decision Making, 15, 313-327.

    Bereby-Meyer, Y., J. Meyer, and D. V. Budescu, (2003), “Decision Making under Internal Uncertainty: The case of Multiple Choice Tests with Different Scoring Rules,” Acta Psychologica, 112, 207-220.

    Bernardo, José M., (1998), “A Decision Analysis Approach to Multiple Choice Examinations,” In: Girón, F. J. (ed.), Applied Decision Analysis, Boston, Kluwer, 195-207.

    Bertrand, M., D. S. Karlan, S. Mullainathan, E. Shafir, and J. Zinman, (2005), “What's Psychology Worth? A Field Experiment in the Consumer Credit Market,” Working Papers 918, Economic Growth Center, Yale University.

    Bredon, G., (2003), “Take-Home Tests in Economics,” Economic Analysis and Policy, Queensland University of Technology, School of Economics and Finance, 33, 52-60.

    Budescu, D. and M. Bar-Hillel, (1993), “To Guess or Not to Guess: A DecisionTheoretic View of Formula Scoring,” Journal of Educational Measurement, 30, 277-291.

    Burgos, A., (2004), “Guessing and Gambling,” Economics Bulletin, 4, 1-10.

    Byrnes, J. P., D. C. Miller, and W. D. Schafer, (1999), “Gender Differences in Risk Taking: A Meta-Analysis,” Psychological Bulletin, 125, 367-383.

    Cadsby, C. B. and E. Maynes, (2005), “Gender, Risk Aversion, and the Drawing Power of Equilibrium in an Experimental Corporate Takeover Game,” Journal of Economic Behavior and Organization, 56, 39-59.

  • Metrics
    No metrics available
Share - Bookmark