A couple of regularized least squares regression models in a feature space are extended by the kernel partial least squares (KPLS) regression model in this paper. PLS is a method based on the projection of input (explanatory) variables to the latent variables (components), and has been developed and established as one of the multivariate statistical process control (MSPC) methods. With PLS, the regression matrix is determined on the basis of a subset of the predictor variables and thus, PLS is able to reduce the number of variables to be considered. In this paper, two kinds of KPLS algorithm for construction of nonlinear regression models in possibly high-dimensional feature spaces are provided. According to the idea of structural risk minimization (SRM), the work described here provides an index of generalization errors. We give the theoretical description of the KPLS algorithms and experimentally compare the algorithms with some existing PLS and KPLS regression models. We will demonstrate that on the data sets employed KPLS achieves better results than PLS and some other KPLS. At the same time, validity of index of generalization errors is proved affirmatively.