We study the contribution of $ν_e \to ν_τ\to τ\to μ$ transitions to the wrong-sign muon sample of the {\em golden} channel of the Neutrino Factory. Muons from tau decays are not really a background, since they contain information from the oscillation signal, and represent a small fraction of the sample. However, if not properly handled they introduce serious systematic error, in particular if the detector/analysis are sensitive to muons of low energy. This systematic effect is particularly troublesome for large $θ_{13} \geq 1^\circ$ and prevents the use of the Neutrino Factory as a precision facility for large $θ_{13}$. Such systematic error disappears if the tau contribution to the golden muon sample is taken into account. The fact that the fluxes of the Neutrino Factory are exactly calculable permits the knowledge of the tau sample due to the $ν_e \to ν_τ$ oscillation. We then compute the contribution to the muon sample arising from this sample {\em in terms of the apparent muon energy}. This requires the computation of a migration matrix $M_{ij}$ which describes the contributions of the tau neutrinos of a given energy $E_i$, to the muon neutrinos of an apparent energy $E_j$. We demonstrate that applying $M_{ij}$ to the data permits the full correction of the otherwise intolerable systematic error.

15 pages, 9 figures, minor changes to match the version to be published on JHEP