In this chapter, we describe three different synthetic datasets that we considered to evaluate the performance of the reviewed recurrent neural network architectures in a controlled environment. The generative models of the synthetic time series are the Mackey–Glass system, NARMA, and multiple superimposed oscillators.Those are benchmark tasks commonly considered in the literature to evaluate the performance of a predictive model. The three forecasting exercises that we study have varying levels of difficulty, given by the nature of the signal and the complexity of the task to be solved by the RNN.