A functional differential equation that arises from the classic theory of neural networks is considered. As the length of the absolute refractory period is varied, there is, as shown here, a super-critical Hopf bifurcation. As the ratio of the refractory period to the time constant of the network increases, a novel relaxation oscillation occurs. Some approximations are made and the period of this oscillation is computed.