Each word to be recognized is represented by hidden Markov models for male and female and an output probability function and a transition probability preset in hidden Markov models for male and female are prestored in a ROM (6). With reference to feature parameters detected by a feature detecting section (3) and the hidden Markov models, a speech recognizing section (4) determines an occurrence probability of a feature parameter sequence. In the process for determining the occurrence probability, the speech recognizing section (4) gives each word one state sequence of a hidden Markov model common to the hidden Markov models for male and female, multiplies an output probability function value by a transition probability of a preset combination among the output probability functions and transition probabilities stored in the ROM (6), selects a maximum product, determines the occurrence probability based on the selected product, and then recognizes the input speech based on the occurrence probability thus determined.