Asian monsoon is influenced by both the tropical Pacific and the Indian Ocean. However, how these oceanic systems impacted the East Asian climate during the Holocene, and their effects on human civilization, remain poorly understood. Here, we present a 15,000-year high-resolution rainfall reconstruction for South China based on multi-proxy records from marine sediments in the northern South China Sea. Our records indicate an anti-phase precipitation pattern between China and southern monsoon regions. We propose that this spatial heterogeneity can be attributed to orbitally driven changes in moisture supply from the Indian Ocean, overprinted by multi-centennial variability linked to the El Niño–Southern Oscillation and solar activity. This moisture configuration profoundly influenced the living environments and subsistence patterns of prehistoric human societies, driving the alternating northward and southward expansions of millet and rice agriculture. Our findings highlight the influence of climatic fluctuations in profoundly shaping human settlement choices and agricultural strategies.