AbstractA recurring challenge in quantum science and technology is the precise control of their underlying dynamics that lead to the desired quantum operations, often described by a set of quantum gates. These gates can be subject to application-specific errors, leading to a dependence of their controls on the chosen circuit, the quality measure and the gate-set itself. A natural solution would be to apply quantum optimal control in an application-oriented fashion. In turn, this requires the definition of a meaningful measure of the contextual gate-set performance. Therefore, we explore and compare the applicability of quantum process tomography, linear inversion gate-set tomography, randomized linear gate-set tomography, and randomized benchmarking as measures for closed-loop quantum optimal control experiments, using a macroscopic ensemble of nitrogen-vacancy centers in diamond as a test-bed. Our work demonstrates the relative trade-offs between those measures and how to significantly enhance the gate-set performance, leading to an improvement across all investigated methods.