Focused on the issue that the robustness of traditional Mel Frequency Cepstral Coefficients (MFCC) feature degrades drastically in speaker recognition system, a kind algorithm that based improved Gammatone Frequency Cepstral Coefficients (GFCC) is proposed. The different between traditional MFCC and GFCC is that GFCC uses Gammatone filter bank to replace Mel filter bank to improve robustness. On this basis, this paper proposes one way that use Multitaper Estimation, MVA (Mean Subtraction, Variance Normzlization and Autoregressive Moving Average Filter)and other technologies to further enhance its robustness and tested with TIMIT speech database. The experimental results show that under different noise and different SNR, the improved GFCC that proposed by this paper has the lowest equal error rate and the best robustness, especially in the noise ratio is lower than 10dB, has greater advantage compared to other algorithms.