Forecasts, models and stress tests are important tools for policymakers and business planners. Recent developments in these related spheres have seen greater emphasis placed on stress tests from a regulatory perspective, while at the same time forecasting performance has been criticized. Given the inter linkages between the two, similar limitations apply to stress tests as to forecasts, and should be borne in mind by practitioners. In addition, the recent evolution of stress tests, and in particular the increasing popularity of scenario-based approaches, raises concerns about how well shortcomings of the associated models are understood. This includes estimated stress cases relative to base cases – the degree of pain – that simple scenario modelling approaches engender. This paper illustrates this phenomenon using simulation techniques, and demonstrates that more extreme stress scenarios need to be employed in order to match the inference from simple value-at-risk approaches. Alternatively, complex modelling approaches can address this concern, but are not widely used to date. Some policymakers, however, seem to be aware of these issues, judging by the severity of some recent stress scenarios.