Here we propose an algorithm, named generalized orthogonal components regression (GOCRE), to explore the relationship between a categorical outcome and a set of massive variables. A set of orthogonal components are sequentially constructed to account for the variation of the categorical outcome, and together build up a generalized linear model (GLM). This algorithm can be considered as an extension of the partial least squares (PLS) for GLMs, but overcomes several issues of existing extensions based on iteratively reweighted least squares (IRLS). First, existing extensions construct a different set of components at each iteration and thus cannot provide a convergent set of components. Second, existing extensions are computationally intensive because of repetitively constructing a full set of components. Third, although they pursue the convergence of regression coefficients, the resultant regression coefficients may still diverge especially when building logistic regression models. GOCRE instead sequentially builds up each orthogonal component upon convergent construction, and simultaneously regresses against these orthogonal components to fit the GLM. The performance of the new method is demonstrated by both simulation studies and a real data example.