Spectro-temporal features have recently shown much performance improvement for robust Automatic Speech Recognition (ASR) tasks. Gabor filters are best known to extract the spectro-temporal cues of speech. Spectro-temporal representation becomes an essential ingredient for two dimensional Gabor based feature extraction methods. State of the art spectro-temporal features is mostly based on Mel spectrogram. However, the time-frequency representation based on the Mel scale is not accurate enough to model the human auditory system. This paper concentrates on obtaining the spectro-temporal representation by incorporating a physiologically and psychoacoustically motivated gammatone filter called gammatonegram. From literature, gammatonegram is found to better approximate the auditory perception of speech. The spectro-temporal features obtained using gammatonegram based Gabor filters are fed to a hybrid Deep Neural Network (DNN)-Hidden Markov Model (HMM) framework to develop the acoustic model of an ASR system. Experimental analysis is carried out with NOISEX-92 database implemented on TIMIT. The experimental results show the better performance gain obtained with the proposed features compared with conventional feature extraction methods.