Abstract Earth system models are indispensable tools in climate studies. The Coupled Model Intercomparison Project (CMIP) is a coordinated effort of the Earth system modeling community to intercompare existing models. An accurate simulation of surface solar radiation fluxes is of major importance for the accuracy of simulations of the near-surface climate in Earth system models. The present study provides a quantitative assessment of the accuracy and multidecadal changes of surface solar radiation fluxes for model results from two phases of CMIP. The entire archives of phase 5 of CMIP (CMIP5) and its predecessor phase 3 (CMIP3) are analyzed for present-day climate conditions. A relative model ranking is provided, and its uncertainty is quantified using different global observational records. It is shown that the choice of an observational dataset can have a major influence on relative model ranking between CMIP models. However the multidecadal variability of surface solar radiation fluxes, also known as global “dimming” or “brightening,” is largely underestimated by the CMIP models.