Since the 1980s, and particularly with the Hopfield model, recurrent neural networks or RNN became a topic of great interest. The first works of neural networks consisted of simple systems of a few neurons that were commonly simulated through analogue electronic circuits. The passage from the equations to the circuits was carried out directly without justification and subsequent formalisation. The present work shows a way to formally obtain the equivalence between an analogue circuit and a neural network and formalizes the connection between both systems. We also show which are the properties that these electrical networks must satisfy. We can have confidence that the representation in terms of circuits is mathematically equivalent to the equations that represent the network.