The performance of population-based multiobjective optimization algorithms is usually evaluated using indicators assessing the quality of the approximation set generated according to convergence, cardinality, spread, and uniformity (the combination of the last two known as diversity). Since not all quality indicators can capture all these properties, we propose to aggregate already-existing indicators into a single measure informing about the algorithm’s performance from a general perspective. To synthesize the desired quality indicators, we build three composite quality indicators (weak, strong, and mixed) based on the reference point approach. This approach enables the use of desirable value ranges for the aggregated quality indicators, defined by aspiration and reservation levels, that allow knowing which algorithms perform better, within, or worse than the desired limits. Each of the composite quality indicators proposed enables a different compensation degree among the aggregated indicators, and their joint use permits a deep insight into the algorithms’ performance. In addition, we show that the weak and mixed composite indicators are Pareto-compliant, and the strong one is weakly Pareto-compliant if at least one of the aggregated indicators is Pareto-compliant. Finally, we demonstrate the benefits of our proposal when comparing many population-based algorithms on three-, five-, and eight-objective optimization problems.

Funding for open access charge: Universidad de Málaga/CBUA.