Nearly 40 years ago, in the pages of this journal, Hubel and Wiesel provided the first description of receptive fields in the primary visual cortex of higher mammals. They defined two classes of cortical cells, ‘simple’ and ‘complex’, based on neural responses to simple visual stimuli. The notion of a hierarchy of receptive fields, where increasingly intricate receptive fields are constructed from more elementary ones, was introduced. Since those early days we have witnessed the birth of quantitative methods to map receptive fields and mathematical descriptions of simple and complex cell function. Insights gained from these models, along with new theoretical concepts, are refining our understanding of receptive field structure and the underlying cortical circuitry. Here, I provide a brief historical account of the evolution of receptive field mapping in visual cortex along with the associated conceptual advancements, and speculate on the shape novel theories of the cortex may take as a result these measurements.