Much research has shown that the definitions of within-class and between-class scatter matrices and regularization technique are the key components to design a feature extraction for small sample size problems. In this paper, we illustrate the importance of another key component, eigenvalue decomposition method, and a new regularization technique was proposed. In the hyperspectral image experiment, the effects of these three components of feature extraction are explored on ill-posed and poorly posed conditions. The experimental results show that different regularization methods need to cooperate with different eigenvalue decomposition methods to reach the best performance, the proposed regularization method, regularized feature extraction (RFE) outperform others, and the best feature extraction for a small sample size classification problem is RFE with nonparametric weighted scatter matrices