Abstract We investigate the origin of the evolution of the population-averaged central stellar mass density (Σ1) of quiescent galaxies (QGs) by probing the relation between stellar age and Σ1 at z ∼ 0. We use the Zurich ENvironmental Study (ZENS), which is a survey of galaxy groups with a large fraction of satellite galaxies. QGs shape a narrow locus in the Σ1–M ⋆ plane, which we refer to as Σ1 ridgeline. Colors of (B − I) and (I − J) are used to divide QGs into three age categories: young (<2 Gyr), intermediate (2–4 Gyr), and old (>4 Gyr). At fixed stellar mass, old QGs on the Σ1 ridgeline have higher Σ1 than young QGs. This shows that galaxies landing on the Σ1 ridgeline at later epochs arrive with lower Σ1, which drives the zeropoint of the ridgeline down with time. We compare the present-day zeropoint of the oldest population at z = 0 with the zeropoint of the quiescent population 4 Gyr back in time, at z = 0.37. These zeropoints are identical, showing that the intrinsic evolution of individual galaxies after they arrive on the Σ1 ridgeline must be negligible, or must evolve parallel to the ridgeline during this interval. The observed evolution of the global zeropoint of 0.07 dex over the last 4 Gyr is thus largely due to the continuous addition of newly quenched galaxies with lower Σ1 at later times (“progenitor bias”). While these results refer to the satellite-rich ZENS sample as a whole, our work suggests a similar age–Σ1 trend for central galaxies.