The Active Appearance Model (AAM) algorithm matches statistical models of shape and texture to images rapidly by assuming a linear relationship between the texture residual and changes in the model parameters. When the texture is represented as raw intensity values, this has been shown to be a reasonable approximation in many cases. However, models built on them are sensitive to changes in illumination conditions. This paper examines the effect of using different representations of image texture to improve the accuracy and robustness of the AAM search. We show that normalising the gradient images by non-linear techniques can give much improved matching with higher accuracy and a wider effective range of convergence. © 2006 IEEE.