Neurotransmission requires the recycling of synaptic vesicles (SVs) to replenish the SV pool, clear release sites, and maintain presynaptic integrity. In spite of decades of research the modes and mechanisms of SV recycling remain controversial. The identification of clathrin-independent modes of SV recycling such as ultrafast endocytosis has added to the debate. Accumulating evidence further suggests that SV membrane retrieval and the reformation of functional SVs are separable processes. This may allow synapses to rapidly restore membrane surface area over a wide range of stimulations followed by slow reformation of release-competent SVs. One of the future challenges will be to pinpoint the exact mechanisms that link SV recycling modes to synaptic activity patterns at different synapses.