We explore the evolution of the cold gas (molecular and neutral hydrogen) of elliptical galaxies and merger remnants ordered into a time sequence on the basis of spectroscopic age estimates. We find that the fraction of cold gas in early merger remnants decreases significantly for ~1-2Gyr, but subsequent evolution toward evolved elliptical systems sees very little change. This trend can be attributed to an initial gas depletion by strong star formation, which subsequently declines to quiescent rates. This explanation is consistent with the merger picture for the formation of elliptical galaxies. We also explore the relation between the HI-to-H2 mass ratio and spectroscopic galaxy age, but find no evidence for a statistically significant trend. This suggests little net HI-to-H2 conversion for the systems in the present sample.