Atrophy of the medial temporal lobe of the brain is key to memory function and memory complaints in old age. While age and some morbidities are major risk factors for medial temporal lobe atrophy, individual differences remain, and mechanisms are insufficiently known. The largest combined neuroimaging and whole genome study to date indicates that medial temporal lobe volume is most associated with common polymorphisms in the GRIN2B gene that encodes for the 2B subunit (NR2B) of the NMDA receptor. Because sleep disruption induces a selective loss of NR2B from hippocampal synaptic membranes in rodents, and because of several other reports on medial temporal lobe sensitivity to sleep disruption, we hypothesized a contribution of the typical age-related increase in sleep-wake rhythm fragmentation to medial temporal lobe atrophy. Magnetic resonance imaging and actigraphy in 138 aged individuals showed that individual differences in sleep-wake rhythm fragmentation accounted for more (19%) of the variance in medial temporal lobe atrophy than age did (15%), or any of a list of health and brain structural indicators. The findings suggest a role of sleep-wake rhythm fragmentation in age-related medial temporal lobe atrophy, that might in part be prevented or reversible.