In this paper, we extend the risk zone concept by creating the Generalized Risk Zone. The Generalized Risk Zone is a model-independent scheme to select key observations in a sample set. The observations belonging to the Generalized Risk Zone have shown comparable, in some experiments even better, classification performance when compared to the use of the whole sample. The main tool that allows this extension is the Cauchy-Schwartz divergence, used as a measure of dissimilarity between probability densities. To overcome the setback concerning pdf's estimation, we used the ideas provided by the Information Theoretic Learning, allowing the calculation to be performed on the available observations only. We used the proposed methodology with Learning Vector Quantization, feedforward Neural Networks, Support Vector Machines, and Nearest Neighbors.