The increasing works on parameter instability, structural changes and regime switches lead to the natural research question whether the assumption of stationarity is appropriate to model volatility processes. Early econometric studies have provided testing procedures of covariance stationarity and have shown empirical evidence for the unconditional time-variation of the dependence structure of many financial time series. After a review of several econometric tests of covariance stationarity, this survey paper focuses on several attempts in the literature to model the time-varying second- order dependence of volatility time series. The approaches that are summarized in this discussion paper propose various specification for this time-varying dynamics. In some of them an explicit variation over time is suggested, such as in the spline GARCH model. Larger classes of nonstationary models have also been proposed, in which the variation of the parameters may be more general such as in the so-called locally stationary models. In another approach that is called “adaptive”, no explicit global model is assumed and local parametric model are adaptively fitted at each point over time. Multivariate extensions are also visited. A comparison of these approaches is proposed in this paper and some illustrations are provided on the two last decades of data of the Dow Jones Industrial Average index.