Exact penalty function algorithms have been employed with considerable success to solve nonlinear semi-infinite programming problems over the last sixteen years. The development of these methods is traced from the perspective of standard nonlinear programming algorithms. The extension of standard theory to the semi-infinite case is illustrated through simple examples and some of the theoretical and computational difficulties are highlighted.