ABSTRACT When used for parameter optimization and/or model selection, traditional mean squared error (MSE)–based measures of forecast accuracy often exhibit a weak or even negative correlation with the economic value of return forecasts measured by, for example, the Sharpe ratios of the resulting portfolios. Deriving a multivariate risk‐adjusted error measure, we show that the RMSE is a special case of this measure under quite restrictive simplifying assumptions. We analyze the contribution of each of these simplifications to the reduction in the explanatory power of the forecast accuracy measure for the shortfall in the attainable Sharpe ratio across a range of well‐known portfolio strategies. We do so both in a Monte Carlo simulation under the assumption of normal i.i.d. returns and in an empirical application.