Liquid crystal spatial light modulators (SLMs) are usually configured and calibrated for phase modulation. However, as they are variable retarders, they also have application as polarization modulators. We show that conventional phase-only calibrations are insufficient for this purpose, and a separate retardance calibration is needed. To overcome this shortcoming we report a simple Twyman-Green interferometer-based setup to realize SLM phase and retardance calibration. For phase calibration, we identify the non-linear, spatially variant response to the drive voltage of the SLM using fringe analysis and both horizontally and vertically polarized incident light. For retardance calibration, we use incident light polarized at 45° and assess the intensity variation. The methods presented are compatible with in situ calibration of SLMs.