AbstractThis paper presents results from simulations based on real data, comparing several competing mean squared error of prediction (MSEP) estimators on principal component regression (PCR) and partial least squares regression (PLSR): leave‐one‐out cross‐validation, K‐fold and adjusted K‐fold cross‐validation, the ordinary bootstrap estimate, the bootstrap smoothed cross‐validation (BCV) estimate and the 0.632 bootstrap estimate. The overall performance of the estimators is compared in terms of their bias, variance and squared error. The results indicate that the 0.632 estimate and leave‐one‐out cross‐validation are preferable when one can afford the computation. Otherwise adjusted 5‐ or 10‐fold cross‐validation are good candidates because of their computational efficiency. Copyright © 2005 John Wiley & Sons, Ltd.