We present an improved kernel recursive least squares (KRLS) algorithm for the online prediction of nonstationary time series. In order to adaptively sparsify a selected kernel dictionary for the KRLS algorithm, the approximate linear dependency (ALD) criterion based KRLS algorithm is combined with the quantized kernel recursive least squares algorithm to provide an initial framework. In order to sufficiently track the strongly changeable dynamic characteristics due to nonstationarity, a forgetting factor is further inserted into the proposed combined algorithm. It is shown that our proposed algorithm, referred to as the FFIKRLS algorithm, provides a clearly compatible algorithm structure and can be improved by the existing modeling techniques from both mapping and weights updating perspectives. Numerical simulations using benchmark Lorenz time series in comparison with existing methods have demonstrated that the proposed algorithm has superior performance in terms of both predictive accuracy and kernel dictionary size.