Echo State Neural Networks (ESN) were applied to forecast the realized variance time series of 19 major stock market indices. Symmetric ESN and asymmetric AESN models were constructed and compared with the benchmark realized variance models HAR and AHAR that approximate the long memory of the realized variance process with a heterogeneous auto-regression. The results show that asymmetric models generally outperform symmetric ones, indicating that a correlation between volatility and returns plays an important role for volatility forecasting. Additionally, models utilizing a logarithmic transformation of the time series achieved generally better results than models applied directly to the realized variance. Echo State Neural Networks outperformed HAR and AHAR models for several important indices (S&P500, DJIA and Nikkei indices), but on average they achieved slightly worse results than the AHAR model. Nevertheless, the results show that Echo State Neural Networks represent an easy-to-use and accurate tool for realized variance forecasting, whose performance may potentially be further improved with meta-parameter optimization.