The legacy impacts of past climates on the current distribution of soil microbial communities are largely unknown. Here, we use data from more than 1,000 sites from five separate global and regional datasets to identify the importance of palaeoclimatic conditions (Last Glacial Maximum and mid-Holocene) in shaping the current structure of soil bacterial communities in natural and agricultural soils. We show that palaeoclimate explains more of the variation in the richness and composition of bacterial communities than current climate. Moreover, palaeoclimate accounts for a unique fraction of this variation that cannot be predicted from geographical location, current climate, soil properties or plant diversity. Climatic legacies (temperature and precipitation anomalies from the present to ~20 kyr ago) probably shape soil bacterial communities both directly and indirectly through shifts in soil properties and plant communities. The ability to predict the distribution of soil bacteria from either palaeoclimate or current climate declines greatly in agricultural soils, highlighting the fact that anthropogenic activities have a strong influence on soil bacterial diversity. We illustrate how climatic legacies can help to explain the current distribution of soil bacteria in natural ecosystems and advocate that climatic legacies should be considered when predicting microbial responses to climate change.