A key question in neuroscience is at which level functional meaning emerges from biophysical phenomena. In most vertebrate systems, precise functions are assigned at the level of neural populations, while single-neurons are deemed unreliable and redundant. Here we challenge this view and show that many single-neuron quantities, including voltages, firing thresholds, excitation, inhibition, and spikes, acquire precise functional meaning whenever a network learns to transmit information parsimoniously and precisely to the next layer. Based on the hypothesis that neural circuits generate precise population codes under severe constraints on metabolic costs, we derive synaptic plasticity rules that allow a network to represent its time-varying inputs with maximal accuracy. We provide exact solutions to the learnt optimal states, and we predict the properties of an entire network from its input distribution and the cost of activity. Single-neuron variability and tuning curves as typically observed in cortex emerge over the course of learning, but paradoxically coincide with a precise, non-redundant spike-based population code. Our work suggests that neural circuits operate far more accurately than previously thought, and that no spike is fired in vain.