In this paper, a fast and effective learning algorithm for training recurrent neural networks, which is realized by introducing and improving the recursive prediction error (RPE) method, is proposed. The improving scheme for RPE algorithm is adding a momentum term in the gradient of Gauss-Newton search direction and using the changeable forgetting factor. Simulation results show that the proposed algorithm achieves far better convergence performance than the classical backpropagation with the momentum term algorithm, and has superior performance compared with the conventional RPE algorithm.