AbstractWhen data are abundant relative to the number of questions asked of them, answers can be formulated using little more than those data. But when data grow more sparse, so too does our tendency to lean on strong models to help us draw inferences. In this research we show how a strong item response model embedded within a fully Bayesian framework allows us to answer two important questions about the reliability and consistency of the clinical diagnosis of hip fractures from very limited data. We also show how the model automatically adjusts diagnoses for biases among the surgeons judging the radiographs. This research illustrates how a Bayesian approach expands the range of problems on which item response models can profitably be used. Copyright © 2009 John Wiley & Sons, Ltd.