While the performance evaluation of judges has become a ubiquitous aspect of modern judicial administration, evaluation mechanisms of are too often utilised uncritically, without reflection on why we evaluate judges, and how ‘measurement’ furthers these objectives. This article provides a conceptual analysis of the role and purpose of performance evaluation, conceiving it as a limited tool of judicial accountability, which itself exists only to promote excellent judging. As such, the efficacy of evaluation mechanisms must always be assessed by reference to their impact on these overarching accountability objectives. The article explores the value of this conception approach by briefly examining three uses of performance evaluation: 1) judicial promotions; 2) judicial retention elections; and 3) judicial professional development. In doing so it illustrates how a clear conceptual approach invites a more nuanced and critical examination of the limitations and benefits of judicial performance evaluation programs. Refereed/Peer-reviewed