Quantization of the electromagnetic field in a class of covariant gauges is performed on a positive metric Hilbert space. Although losing manifest covariance, we find at the free field level the existence of two physical spaces where Poincaré transformations are implemented unitarily. This gives rise to two different physical interpretations of the theory. Unitarity of the S operator for an interaction with an external source then forces one to postulate that a restricted gauge invariance must hold. This singles out one interpretation, the one where two transverse photons are physical.