Red emitting Mn4+-doped crystalline materials have potential for application in light emitting devices and therefore it is important to understand how the optical properties of Mn4+ are influenced by the host lattice the Mn4+ ions are situated in. In this work we investigate the effect of the host cations in the second coordination sphere on the Mn4+ emission by studying the luminescence of Mn4+ ions doped into three isostructural rare earth (RE) stannate RE2Sn2O7 pyrochlores (RE3+ = Y3+, Lu3+ or Gd3+). It is found that the energies of the Mn4+4T1 and 4T2 states significantly increase with decreasing Mn4+-O2- distance, whereas the energy of the 2E level shows a small shift to higher energies from RE3+ = Gd3+ to Lu3+ to Y3+. The observed trend for the 2E level energy is not related to the size of the RE3+ ion and is not in line with theoretical calculations reported previously. Low temperature emission spectra of the RE2Sn2O7:Mn4+ phosphors reveal that only asymmetrical vibronic modes couple to the 2E → 4A2 transition and furthermore show there is significant and unexpected local disorder for Mn4+ in Gd2Sn2O7 that is not observed for Mn4+ in the other hosts. Photoluminescence decay measurements demonstrate that the luminescence of RE2Sn2O7:Mn4+ is strongly quenched below room temperature which is assigned to non-radiative relaxation via a low-lying O2− → Mn4+ charge-transfer state.