A wide range of issues involving sequence data have been solved using recurrent neural networks with various types of hidden units. One of the most recent proposals, neural circuit policies (NCP), has shown comparable promising results on our example datasets. In this paper, we compare three model variants of recurrent neural networks (RNN) with neural circuit policies (NCP), which is a subclass of continuous-time RNNs, with varying neuronal time-constant realized by their nonlinear synaptic transmission model. These four model variants, Deep RNN, LSTM, GRU, and NCP, were tested on synthetic sinusoidal training example sequences and the navigation of a robot dataset. The NCP model has shown similar performance to the SimpleRNN model while using fewer parameters and thus lowering training expense, and thus may be used as an alternate to the SimpleRNN recurrent neural network. However, in terms of lowest loss, LSTM and GRU performed slightly better than SimpleRNN and NCP on the datasets considered.