We model the size distribution of supernova remnants to infer the surrounding ISM density. Using simple, yet standard SNR evolution models, we find that the distribution of ambient densities is remarkably narrow; either the standard assumptions about SNR evolution are wrong, or observable SNRs are biased to a narrow range of ambient densities. We show that the size distributions are consistent with log-normal, which severely limits the number of model parameters in any SNR population synthesis model. Simple Monte Carlo simulations demonstrate that the size distribution is indistinguishable from log-normal when the SNR sample size is less than 600. This implies that these SNR distributions provide only information on the mean and variance, yielding additional information only when the sample size grows larger than $\sim{600}$ SNRs. To infer the parameters of the ambient density, we use Bayesian statistical inference under the assumption that SNR evolution is dominated by the Sedov phase. In particular, we use the SNR sizes and explosion energies to estimate the mean and variance of the ambient medium surrounding SNR progenitors. We find that the mean ISM particle density around our sample of SNRs is $��_{\log{n}} = -1.33$, in $\log_{10}$ of particles per cubic centimeter, with variance $��^2_{\log{n}} = 0.49$. If interpreted at face value, this implies that most SNRs result from supernovae propagating in the warm, ionized medium. However, it is also likely that either SNR evolution is not dominated by the simple Sedov evolution or SNR samples are biased to the warm, ionized medium (WIM).