During the last decade, the use of job exposure matrices (JEM) in occupational studies has grown but uncertainty remains among epidemiologists and industrial hygienists regarding this methodology and appropriate methodological tools are needed to study the performance of JEM. It must first be emphasized that a true validation can rarely be achieved and that the performance of a JEM must be tested against the exposure evaluation of another method taken as a reference, such as expert assessment. This comparison comprises three main elements: the ability of the JEM to evaluate accurately the exposure itself, its statistical performances in terms of bias and power and its ability to detect known associations between risk factors and disease. When comparing JEM and experts, all aspects of the two methods have to be looked at and a balance struck between the advantages and shortcomings of each of them. The problem should not be reduced to a trade-off between the precision provided by experts and the cost savings possible by using JEM. Finally, it is important to avoid a systematic opposition between JEM and expert assessments, as one positive result of their comparison may be the improvement of both methods of exposure evaluation.