The neuron machine (NM) is a hardwarearchitecture that can be used to design efficient neural networksimulation systems. However, owing to its intrinsicunidirectional nature, NM architecture does not supportbackpropagation (BP) learning algorithms. This paperproposes novel schemes for NM architecture to support BPalgorithms. Reverse-mapping memories, synapse placementalgorithm, and a memory structure called triple rotatememory can be used to share synaptic weights in both the feedforwardand error BP stages without degrading thecomputational performance. An NM system supporting a BPtraining algorithm was implemented on a field-programmablegate array board and successfully trained a neural networkthat can classify MNIST handwritten digits. The implementedsystem showed a better performance over most chip-level orboard-level systems based on other hardware architectures.