At fixed stellar mass, the population of passive galaxies has increased its mean effective radius < Re > by a factor ~5 in the last 10 Gyr, decreasing its mean stellar mass density (S = Mstar/(2πRe 2 ) by a factor >> 10. Whether this increase in < Re > is mainly due to the size-growth of individual galaxies through dry mergers, or to the fact that newly quenched galaxies have a larger size, is still matter of debate. A promising approach to shed light on this issue is to investigate the evolution of the number density of passive galaxies as a function of their mass density. In this context, massive (Mstar >10^11 Msun) passive galaxies are the most intriguing systems to study, since, in a hierarchical scenario, they are expected to accrete their stellar mass mainly by mergers. The wide area (∼ 16 sq. deg) and high sampling rate (∼ 40%) of the spectroscopic survey VIPERS allowed us to collect a sample of ∼ 2000 passive massive galaxies over the redshift range 0.5 < z < 1.0 and to study, with unprecedented statistics, the evolution of their number density as function of their mean stellar mass density in this redshift range. Taking advantage of both spectroscopic (D4000) and photometric (SED fitting) data available, we studied the age of the stellar population of passive galaxies as function both of redshift and mass density. This information, combined with the evolution of the number density allowed us to put constraints on the mass accretion scenarios of passive galaxies. In this talk I will present our results and conclusions and how they depend on the environment in which the galaxies reside.