publication . Preprint . 2015

A Survey: Time Travel in Deep Learning Space: An Introduction to Deep Learning Models and How Deep Learning Models Evolved from the Initial Ideas

Wang, Haohan; Raj, Bhiksha;
Open Access English
  • Published: 16 Oct 2015
This report will show the history of deep learning evolves. It will trace back as far as the initial belief of connectionism modelling of brain, and come back to look at its early stage realization: neural networks. With the background of neural network, we will gradually introduce how convolutional neural network, as a representative of deep discriminative models, is developed from neural networks, together with many practical techniques that can help in optimization of neural networks. On the other hand, we will also trace back to see the evolution history of deep generative models, to see how researchers balance the representation power and computation comple...
arXiv: Quantitative Biology::Neurons and CognitionComputer Science::Neural and Evolutionary Computation
free text keywords: Computer Science - Learning, Computer Science - Neural and Evolutionary Computing
Download from

[37] OJA, E., AND KARHUNEN, J. On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix. Journal of mathematical analysis and applications 106, 1 (1985), 69-84. [OpenAIRE]

[38] ORPONEN, P. Computational complexity of neural networks: a survey. Nordic Journal of Computing 1, 1 (1994), 94-110. [OpenAIRE]

[39] PASCANU, R., GULCEHRE, C., CHO, K., AND BENGIO, Y. How to construct deep recurrent neural networks. arXiv preprint arXiv:1312.6026 (2013). [OpenAIRE]

[40] PERSON, K. On lines and planes of closest fit to system of points in space. philiosophical magazine, 2, 559-572, 1901.

Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue