The electronic structures of Pb and non-Pb containing phosphates, specifically ${\mathrm{Pb}}_{3}{(\mathrm{P}{\mathrm{O}}_{4})}_{2}$, ${\mathrm{Pb}}_{2}{\mathrm{P}}_{2}{\mathrm{O}}_{7}$, $\mathrm{Sc}\mathrm{P}{\mathrm{O}}_{4}$, $\mathrm{Y}\mathrm{P}{\mathrm{O}}_{4}$, and $\mathrm{Lu}\mathrm{P}{\mathrm{O}}_{4}$, are compared using a combination of density functional calculations and optical measurements. We find that the characteristic difference between these two classes is that the band gaps in the Pb containing materials are significantly lower because of the presence of Pb $6p$ derived bands in the gap between the valence bands and the P-O antibonding states. This may be favorable from the point of view of energy transfer in scintillator applications of Pb-based phosphates. However, it also suggests that different activators than those used in other phosphate scintillators may be useful to increase the efficiency.