A purely theoretical approach has been found to be of limited value in the solution of practical Pattern Recognition problems. Difficulties arise when relating infinite mathematics to reality, e.g. “algorithmic convergence” must be replaced by a vaguer notion of “satisfactory performance”. Experimentation has been used to study this and related problems: a) Learning in noise; b) Similarity of classifiers; c) Instability of classifiers; d) Relating infinite‐sample analysis to finite data sets (reference to pdf estimation). Finally, the system requirements for effective experimentation are discussed.