AbstractOverconfidence is generally regarded as one of the most robust findings in the psychology of judgment. A precise method for evaluating overconfidence is essential if researchers are to validate these findings. Although peer‐comparison questions are a convenient tool for measuring overconfidence, their validity has been questioned. We employed a specific paradigm to verify the validity, and the respondents were asked to predict a verifiable future event in a real‐world setting that allowed empirical checking and comparison between the actual result and the prediction. Studies 1 and 2 found that the actual percentile of overconfidence could be accurately predicted using our initial calculation of participants’ peer‐comparison overconfidence in answering questions about academic performance. Study 3 found a similar effect when using questions related to job hunting. All studies indicated that peer‐comparison questions are valid for measuring bias in self‐evaluation. Thus, future studies could employ peer‐comparison questions to investigate the domain specificity versus the domain generality of overconfidence.