AbstractWe propose a highly flexible sequential methodology for the experimental analysis of optimization algorithms. The proposed technique employs computational statistic methods to investigate the interactions among optimization problems, algorithms, and environments. The workings of the proposed technique are illustrated on the parameterization and comparison of both a population–based and a direct search algorithm, on a well–known benchmark problem, as well as on a simplified model of a real–world problem. Experimental results are reported and conclusions are derived. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)