arxiv: Computer Science::Neural and Evolutionary Computation
The backpropagation algorithm for calculating gradients has been widely used in computation of weights for deep neural networks (DNNs). This method requires derivatives of objective functions and has some difficulties finding appropriate parameters such as learning rate... View more
1. Bengio,Y., Lamblin, P., Popovici, D., and Larochelle, H.: Greedy layer-wisetraining of deep networks. In Proc. Advances in Neural Information Processing Systems 19 153-160 (2006).
2. Ciresan, D.C., Meier, U., Masci, J., Gambardella, L.M., and Schmidhuber, J.: Flexible, high performance convolutional neural networks for image classification, Proc. 22nd International Joint Conference on Artificial Intelligence, 1237-1242 (2011).
3. Ding, D., Li, T., and Jordan, M. I.: Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 45-55 (2010).
4. Glorot, X. and Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks, in International conference on artificial intelligence and statistics, 249-256 (2010).
5. Glorot, X., Bordes, A., and Bengio., Y.: Deep sparse rectifier neural networks. In Proc.14th International Conference on Artificial Intelligence and Statistics 315-323 (2011).
6. Hinton, G.E., Deng, L., Yu, D., Dahl, G.E., Mohamed, A., Jaitly, N., Senior, A., and Vanhoucke, V.: Deep neural networks for acoustic modeling in speech recognition, IEEE Signal Processing Magazine 29,82-97 (2012).
7. Kingma, D. P. and Ba, J.: ADAM: A Method for Stochastic Optimization, The International Conference on Learning Representations (ICLR), San Diego, 2015
8. Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Computer Science Department, University of Toronto, Tech. Rep, 1 7 (2009).
9. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., and Jackel, L.D.: Backpropagation applied to handwritten zip code recognition, Neural Computation 1, 541-551 (1989).
10. LeCun, Y.: The MNIST database of handwritten digits, http://yann.lecun.com/exdb/mnist.