Does evolution proceed faster in larger or smaller populations? The relationship between effective population size (Ne) and the rate of evolution has consequences for our ability to understand and interpret genomic variation, and is central to many aspects of evolution and ecology. Many factors affect the relationship between Ne and the rate of evolution, and recent theoretical and empirical studies have shown some surprising and sometimes counterintuitive results. Some mechanisms tend to make the relationship positive, others negative, and they can act simultaneously. The relationship also depends on whether one is interested in the rate of neutral, adaptive, or deleterious evolution. Here, we synthesize theoretical and empirical approaches to understanding the relationship and highlight areas that remain poorly understood.