publication . Preprint . 2017

Deep Echo State Network (DeepESN): A Brief Survey

Gallicchio, Claudio; Micheli, Alessio;
Open Access English
  • Published: 12 Dec 2017
Abstract
The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced Deep Echo State Network (DeepESN) model opened the way to an extremely efficient approach for designing deep neural networks for temporal data. At the same time, the study of DeepESNs allowed to shed light on the intrinsic properties of state dynamics developed by hierarchical compositions of recurrent layers, i.e. on the bias of depth in RNNs architectural design. In this paper, we summarize the advancements in the development, analysis and applications of De...
Subjects
free text keywords: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Statistics - Machine Learning
Download from
37 references, page 1 of 3

[1] D. Verstraeten, B. Schrauwen, M. d'Haene, D. Stroobandt, An experimental unification of reservoir computing methods, Neural networks 20 (3) (2007) 391-403. [OpenAIRE]

[2] M. Lukoˇseviˇcius, H. Jaeger, Reservoir computing approaches to recurrent neural network training, Computer Science Review 3 (3) (2009) 127-149.

[3] W. Maass, T. Natschl¨ager, H. Markram, Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural computation 14 (11) (2002) 2531-2560.

[4] P. Tinˇo, G. Dorffner, Predicting the future of discrete sequences from fractal representations of the past, Machine Learning 45 (2) (2001) 187-217.

[6] H. Jaeger, The ”echo state” approach to analysing and training recurrent neural networks - with an erratum note, Tech. rep., GMD - German National Research Institute for Computer Science, Tech. Rep. (2001).

[7] P. Tinˇo, B. Hammer, M. Bod´en, Markovian bias of neural-based architectures with feedback connections, in: Perspectives of neural-symbolic integration, Springer, 2007, pp. 95-133.

[8] C. Gallicchio, A. Micheli, Architectural and markovian factors of echo state networks, Neural Networks 24 (5) (2011) 440-456. [OpenAIRE]

[9] I. B. Yildiz, H. Jaeger, S. J. Kiebel, Re-visiting the echo state property, Neural networks 35 (2012) 1-9.

[10] G. Manjunath, H. Jaeger, Echo state property linked to an input: Exploring a fundamental characteristic of recurrent neural networks, Neural computation 25 (3) (2013) 671-696.

[11] M. Massar, S. Massar, Mean-field theory of echo state networks, Physical Review E 87 (4) (2013) 042809. [OpenAIRE]

[12] P. Tinˇo, Fisher memory of linear wigner echo state networks, in: Proceedings of the 25th European Symposium on Artificial Neural Networks (ESANN), i6doc.com, 2017, pp. 87-92.

[13] M. Lukoˇseviˇcius, H. Jaeger, B. Schrauwen, Reservoir computing trends, KI-Ku¨nstliche Intelligenz 26 (4) (2012) 365-371.

[14] B. Schrauwen, D. Verstraeten, J. Van Campenhout, An overview of reservoir computing: theory, applications and implementations, in: Proceedings of the 15th European Symposium on Artificial Neural Networks. p. 471-482 2007, 2007, pp. 471-482.

[15] F. Palumbo, C. Gallicchio, R. Pucci, A. Micheli, Human activity recognition using multisensor data fusion based on reservoir computing, Journal of Ambient Intelligence and Smart Environments 8 (2) (2016) 87-107.

[16] E. Crisostomi, C. Gallicchio, A. Micheli, M. Raugi, M. Tucci, Prediction of the italian electricity price for smart grid applications, Neurocomputing 170 (2015) 286-295. [OpenAIRE]

37 references, page 1 of 3
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue