We currently lack a unified and mechanistic account of how the hippocampus supports a range of disparate cognitive functions that includes episodic memory, imagining the future, and spatial navigation. Here, we argue that in order to leverage this long-standing issue, traditional notions regarding the architecture of memory should be eschewed. Instead, we invoke the idea that scenes are central to hippocampal information processing. This view is motivated by mounting evidence that the hippocampus is constantly constructing spatially coherent scenes, automatically anticipating and synthesizing representations of the world beyond the immediate sensorium. By characterizing the precise relationship between scenes and the hippocampus, we believe a theoretically enriched understanding of its fundamental role and its breakdown in pathology can emerge.