Temporal Regularized Learning (TRL) is a highly local and self-supervised prodecure that optimizeseach neuron individually. We adapt the self-supervised loss formulation of VICReg, consistingof variance, invariance and covariance to input streams with sequential coherence and for online-compatibility. It removes the need for biphasic updates, negatives or inner-loop convergence, giventhree scalar memory units per neuron and an auxiliary lateral network. Knowledge about downstreamtasks can be injected through the sequence ordering, allowing for supervised training. We presentTRL and its simplified variant, TRL-S. Experiments on MNIST show TRL is competetive withbackpropagation, Forward-Forward and Equilibrium Propagation, while TRL-S achieves similarperformance despite its simplified setup. We show TRL creates neurons with specialized receptivefields at the first layer. In later layers, some neurons specialize by activating only for some types ofinput. This upload contains a zipped version of the repository.