A speech recognition system and method that dynamically switches between reference patterns based on training information produced under different ambient noise levels to enhance speech recognition accuracy, is presented herein. In accordance with an embodiment of the invention, the speech recognition system includes a speech capturing device configured to capture an input utterance and a speech recognition processing mechanism configured to process the input utterances and to generate an identified utterance signal representing a recognized utterance. The system further includes a sensor configured to detect a plurality of ambient noise levels and to supply a detected ambient noise level to the speech recognition processing mechanism and a speech model containing a plurality of stored reference pattern sets representing utterances to be recognized. Each of the stored reference pattern sets are based on training information corresponding to a particular ambient noise levels. As such, in response to receiving the input utterance and detected ambient noise level, the speech recognition processing mechanism switches to the stored reference pattern set corresponding to the detected ambient noise level and determines a recognized utterance by comparing the input utterance to the utterances contained in the corresponding stored reference pattern set. The speech recognition processing mechanism then generates a corresponding identified utterance signal, indicating a recognized utterance, which is applied to related applications to execute predetermined tasks.