With the recent availability of high-frequency financial data the long range dependence of volatility regained researchers' interest and has lead to the consideration of long memory models for realized volatility. The long range diagnosis of volatility, however, is usually stated for long sample periods, while for small sample sizes, such as e.g. one year, the volatility dynamics appears to be better described by short-memory processes. The ensemble of these seemingly contradictory phenomena point towards short memory models of volatility with nonstationarities, such as structural breaks or regime switches, that spuriously generate a long memory pattern (see e.g. Diebold and Inoue, 2001; Mikosch and Starica, 2004b). In this paper we adopt this view on the dependence structure of volatility and propose a localized procedure for modeling realized volatility. That is at each point in time we determine a past interval over which volatility is approximated by a local linear process. Using S&P500 data we find that our local approach outperforms long memory type models in terms of predictability.