A new paradigm for multivariate regression is proposed; principal regression analysis (PRA). It entails learning a low dimensional subspace over sample-specific regressors. For a given input, the model predicts a subspace thought to contain the corresponding response. Using this subspace as a prior, the search space is considerably more constrained. An efficient local optimisation strategy is proposed for learning and a practical choice for its initialisation suggested. The utility of PRA is demonstrated on the task of non-rigid face and car alignment using challenging "in the wild" datasets, where substantial performance improvements are observed over alignment with a conventional prior.