Prior studies have repeatedly shown that probabilistic seismic hazard maps from several different countries predict higher shaking than that observed. Previous map assessments have not, however, considered the influence of site response on hazard. Seismologists have long acknowledged the influence of near-surface geology, in particular low-impedance sediment layers, on earthquake ground-motion at frequencies of engineering concern. Although the overall effects of site response are complex, modern ground-motion models (GMMs) account for site effects using terms based on VS30, the time-averaged shear-wave velocity in the upper 30 m of the Earth’s surface. In this study, we consider general implications of incorporating site terms from modern GMMs using site-specific VS30 as a proxy in probabilistic seismic hazard maps for California. At the long periods (1–5 s) that affect tall buildings, site terms amplify the mapped hazard by factors of 1–3 at many sites relative to maps calculated for the standard reference soft-rock site condition, VS30 = 760 m/s. However, at the short periods of ground-motion that are the main contributors to peak ground acceleration (PGA) and thus affect smaller structures, only negligible effects occur due to nonlinear deamplification of strong ground-motion at high frequencies. Nonlinear deamplification increases as the shaking level increases. For very strong shaking, deamplification can overcome the linear amplification, yielding net deamplification. We explore the implications of these results for the evaluation of hazard maps. Because site effects do not change the maps appreciably at short periods, we can exclude site response as an explanation for why the maps overpredict historically observed shaking as captured by the California Historical Intensity Mapping Project (CHIMP) dataset. The results are expected to be generalizable to regions that are comparable to California in terms of structure and seismicity rates. In low-to-moderate-seismicity regions where the hazard reflects weaker shaking, nonlinear site response is expected to be less important for the hazard.