The efficiency of single-junction solar cells is limited to about 30% (the Shockley-Queisser limit). Spectral mismatch losses (transparency to low-energy photons, thermalization of high-energy photons) strongly contribute to lowering the maximum efficiency. To reduce thermalization losses, photon splitting is proposed and observed for a variety of lanthanide-doped materials. For Er3+, even a one-to-three photon-splitting process has been reported, yielding three IR photons at around 1530 nm following absorption of one blue-green photon. This is especially beneficial for narrow band gap solar cells, such as crystalline Ge. Here, we report on photon splitting for Er3+ in YVO4. Following absorption in the 2H11/2 and 4S3/2 levels (520-550 nm), efficient cross-relaxation (CR) yields two excited Er3+ ions: one in the 4I9/2 state and one in the 4I13/2 state (CR1). A second CR step from the 4I9/2 state, leaving both Er3+ ions in the 4I13/2 excited state (CR2), is crucial in realizing efficient three IR photon splitting. It is demonstrated here that the second step has a low efficiency, as a result of competing fast multiphonon relaxation, 4I9/2→4I11/2, and a large energy mismatch, which makes the CR2 step thermally activated. Based on experiments and theory, a maximum quantum efficiency of 170% is calculated for IR emission, following blue-green excitation in YVO4:Er3+. An outlook is presented for three-photon splitting in low-phonon-energy hosts, where nonradiative multiphonon relaxation is suppressed. The anti-Stokes nature of the second CR step makes three-photon splitting unlikely and prevents the realization of IR quantum yields above 200%.