We present a simple methodology for modeling the time variation in volatilities and other higher-order moments using a recursive updating scheme that is similar to the familiar RiskMetrics approach. The parameters are updated using the score of the forecasting distribution, which allows the parameter dynamics to adapt automatically to any non-normal data features, and increases the robustness of the subsequent estimates. The new approach nests several of the earlier extensions to the exponentially weighted moving average (EWMA) scheme. In addition, it can be extended easily to higher dimensions and alternative forecasting distributions. The method is applied to Value-at-Risk forecasting with (skewed) Student's t distributions and a time-varying degrees of freedom and/or skewness parameter. We show that the new method is as good as or better than earlier methods for forecasting the volatility of individual stock returns and exchange rate returns.