This article proposes a mathematical-programming-based approach to solve the classification problem in discriminant analysis which explicitly considers the classification gap. The procedure consists of two distinct phases and initially treats the classification gap as a fuzzy set in which the classification rule is not yet established. The nature of the classification gap is examined and a variety of methods are discussed which can be applied to identify the most appropriate classification rule over the fuzzy set. The proposed methodology has several potential advantages. First, it offers a more refined approach to the classification problem, facilitating careful analysis of the fuzzy region where the classification decision may not be obvious. Secondly, the two-phase approach enables the analysis of larger data sets when using computer-intensive procedures such as mixed-integer programming. Finally, because of the restricted choice of separating hyperplanes in phase 2, the approach appears to be more robust than other classification techniques with respect to outlier-contaminated data conditions. The robustness issue and computational advantage of our proposed methodology are illustrated using a limited simulation experiment.