Accurate states and unknown random bias estimation for well- and ill-conditioned systems are crucial for several applications. In this paper, a fusion of a two-stage Kalman filter and an information filter, and its extensions are considered to estimate the state variables and unknown random bias. Specifically, we propose four extensions of two-stage Kalman filters: two-stage information filter (TSIF), multi-sensor two-stage information filter (M-TSIF) and their square-root versions. The TSIF deals with single-sensor systems whereas the M-TSIF is capable to handle multi-sensor systems. For ill-conditioned systems, numerically stable square-root versions of TSIF and M-TSIF are developed. The performance of the proposed filters (along with the existing two-stage Kalman filter), for well- and ill-conditioned cases, is demonstrated on a quadruple-tank model.