To determine the accuracy of the SF-36 summary mental and physical health scales in reflecting their underlying subscales using the traditional method of scoring based on factor coefficients derived through principle components analysis and orthogonal rotation. A representative Australian population survey containing the SF-36 was used to obtain factor coefficients from principle components analysis and orthogonal rotation for scoring the physical component summary (PCS) and the mental component summary (MCS) of the SF-36 in the traditional way. In addition two other methods were used to produce coefficients. The first method used maximum likelihood extraction and oblique rotation. The second method fit a structural equation model to the data in a confirmatory factor analysis. The coefficients derived by each of the methods were applied to the data of a second representative population survey. This survey also provided data on physical and mental health status which allowed comparison of the summary scores and underlying subscales according to various health states. Neither of the scoring methods based on the exploratory factor analyses methods (orthogonal and oblique) produced summary scale scores, by age group, that adequately reflected the underlying subscales. When coefficients derived using structural equation modeling were fit to the data in a confirmatory factor analysis the MCS and PCS accurately reflected their underlying subscale scores. They also produced MCS and PCS scores for the various health states as would be expected from the underlying subscales. The traditional methods of scoring the SF-36 summary scales produce results that would not be expected from the underlying subscales. The problem was only corrected by fitting a structural equation model to the data in a confirmatory factor analysis. The results advise caution in the use of the SF-36 summary scales and suggests that alternative methods of developing factor coefficients need to be employed in studies using the SF-36 summary scales.