This paper considers the issue of forecasting financial fragility of banks and insurances using a panel data set of performance indicators, namely distance-to-default, taking unobserved common factors into account. We show that common factors are important in the performance of banks and insurances, analyze the influences of a number of observable factors on banking and insurance performance, and evaluate the forecasts from our model. We find that taking unobserved common factors into account reduces the the root mean square forecasts error of firm specific forecasts by up to 11% and of system forecasts by up to 29% relative to a model based only on observed variables. Estimates of the factor loadings suggest that the correlation of financial institutions has been relatively stable over the forecast period.