A simple model of quasar number density evolution is presented based on the occurrence of quasar-like radio galaxies (i.e., strong optical emission lines and type 2 radio morphology) exclusively in regions of low galaxy and intergalactic medium (IGM) density. This suggests a limit for the IGM density of 10 to the -4th (+ or - 1) per cu cm below which quasars are allowed to form and above which they are not allowed. In the recent past (z not greater than 1), the inferred quasar environments are the outskirts of clusters and near the centers of groups of galaxies. However, models of rich cluster evolution consistent with current X-ray observations predict gas densities of less than 10 to the -4th per cu cm in cluster cores in the more distant past (z between 1 and 5). This suggests that quasars were allowed to form in the cores of rich clusters at those epochs, which explains both the rich absorption spectra of high-redshift quasars and the absence of clusters surrounding quasars at lower redshift.