This paper evaluates the model risk of models used for forecasting systemic and market risk. Model risk, which is the potential for different models to provide inconsistent outcomes, is shown to be increasing with and caused by market uncertainty. During calm periods, the underlying risk forecast models produce similar risk readings, hence, model risk is typically negligible. However, the disagreement between the various candidate models increases significantly during market distress, with a no obvious way to identify which method is the best. Finally, we discuss the main problems in risk forecasting for macro prudential purposes and propose an evaluation criteria for such models.