Item Response Modeling of Forced-Choice Questionnaires

Article English OPEN
Brown, Anna ; Maydeu-Olivares, Alberto (2011)

Multidimensional forced-choice formats can significantly reduce the impact of numerous response biases typically associated with rating scales. However, if scored with classical methodology, these questionnaires produce ipsative data, which lead to distorted scale relationships and make comparisons between individuals problematic. This research demonstrates how item response theory (IRT) modeling may be applied to overcome these problems. A multidimensional IRT model based on Thurstone’s framework for comparative data is introduced, which is suitable for use with any forced-choice questionnaire composed of items fitting the dominance response model, with any number of measured traits, and any block sizes (i.e., pairs, triplets, quads, etc.). Thurstonian IRT models are normal ogive models with structured factor loadings, structured uniquenesses, and structured local dependencies. These models can be straightforwardly estimated using structural equation modeling (SEM) software Mplus. A number of simulation studies are performed to investigate how latent traits are recovered under various forced-choice designs and provide guidelines for optimal questionnaire design. An empirical application is given to illustrate how the model may be applied in practice. It is concluded that when the recommended design guidelines are met, scores estimated from forced-choice questionnaires with the proposed methodology reproduce the latent traits well.
  • References (18)
    18 references, page 1 of 2

    Hogan, R. (1983). A socioanalytic theory of personality. In M.M. Page (Ed.), Nebraska Symposium on Motivation (pp. 336-355). Lincoln: University of Nebraska Press.

    Maydeu-Olivares, A. (1999). Thurstonian modeling of ranking data via mean and covariance structure analysis. Psychometrika, 64, 325-340.

    Maydeu-Olivares, A. & Böckenholt, U. (2005). Structural equation modeling of paired comparisons and ranking data. Psychological Methods, 10, 285-304.

    Maydeu-Olivares, A. & Coffman, D. L. (2006). Random intercept item factor analysis. Psychological Methods, 11, 344-362.

    McCloy, R., Heggestad, E., Reeve, C. (2005). A Silk Purse From the Sow's Ear: Retrieving Normative Information From Multidimensional Forced-Choice Items. Organizational Research Methods, 8, 222-248.

    McDonald, R.P. (1999). Test theory. A unified approach. Mahwah, NJ: Lawrence Erlbaum.

    Meade, A. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organisational Psychology, 77, 531-552.

    Muthén, L.K. & Muthén, B. (1998-2007). Mplus 5. Los Angeles, CA: Muthén & Muthén.

    Reckase, M. (2009). Multidimensional Item Response Theory. New York: Springer.

    Samejima, F. (1969). Calibration of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17.

  • Similar Research Results (4)
  • Metrics
    0
    views in OpenAIRE
    0
    views in local repository
    94
    downloads in local repository

    The information is available from the following content providers:

    From Number Of Views Number Of Downloads
    Kent Academic Repository - IRUS-UK 0 94
Share - Bookmark