We exploit the tools of differential geometry to provide a clear explanation for the finite sample lack of invariance of the Wald statistic to algebraically equivalent reformulations of the null hypothesis. The Wald statistic is shown, in general, to be an improper geometric quantity and hence is not invariant to reparametrizations. The geometric approach also suggests an alternative invariant test based on the calculation of geodesic distances in curved manifolds. We show how this ``Fisher geodesic statistic'' may be easily calculated and applied in the case of testing nonlinear restrictions in the general linear model and also when it will coincide with the Wald statistic. We are also able to extend the familiar inequalities relating the Wald, score, and likelihood ratio statistics to the nonliner case with the fundamental difference that the Fisher geodesic takes the place previously occupied by the Wald statistic in the relevant inequality. The paper also provides an introduction to the methods of differential geometry (the relevant concepts are briefly summarized in the Appendix) and hopefully demonstrates its potential for econometricians.