publication . Preprint . 2017

A Deep Convolutional Auto-Encoder with Pooling - Unpooling Layers in Caffe

Turchenko, Volodymyr; Chalmers, Eric; Luczak, Artur;
Open Access English
  • Published: 18 Jan 2017
Abstract
This paper presents the development of several models of a deep convolutional auto-encoder in the Caffe deep learning framework and their experimental evaluation on the example of MNIST dataset. We have created five models of a convolutional auto-encoder which differ architecturally by the presence or absence of pooling and unpooling layers in the auto-encoder's encoder and decoder parts. Our results show that the developed models provide very good results in dimensionality reduction and unsupervised clustering tasks, and small classification errors when we used the learned internal code as an input of a supervised linear classifier and multi-layer perceptron. T...
Subjects
ACM Computing Classification System: Data_CODINGANDINFORMATIONTHEORY
free text keywords: Computer Science - Neural and Evolutionary Computing, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Learning
Download from
28 references, page 1 of 2

[1] D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back-propagating errors, Nature. 323 (1986) 533-536. [OpenAIRE]

[2] Y. LeCun, Modeles connexionistes de l'apprentissage, Ph.D. thesis, Universite de Paris VI, 1987.

[3] H. Bourland, Y. Kamp, Auto-association by multilayer perceptrons and singular value decomposition, Biological Cybernetics. 59 (1988) 291-294. [OpenAIRE]

[4] P. Baldi, K. Hornik, Neural networks and principal component analysis: Learning from examples without local minima, Neural Networks. 2 (1989) 53-58. [OpenAIRE]

[5] G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science. 313 (2006) 504-507.

[6] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. of the IEEE. 86 (11) (1998) 2278-2324.

[7] Cuda-convnet2, High-performance C++/CUDA implementation of convolutional neural networks. https://github.com/akrizhevsky/cuda-convnet2, 2014 (accessed 23.08.2016).

[8] Theano Development Team, Theano: A Python framework for fast computation of mathematical expressions, arXiv: 1605.02688, 2016.

[9] Lasagne, Lasagne is a lightweight library to build and train neural networks in Theano. http://lasagne.readthedocs.org/, 2014 (accessed 23.08.2016).

[10] Keras: Deep learning library for Theano and TensorFlow. https://keras.io/, 2015 (accessed 23.08.2016).

[11] R. Collobert, K. Kavukcuoglu, C. Farabet, Torch7: A Matlab-like environment for machine learning, in: J. Shawe-Taylor, R.S. Zemel, P.L. Bartlett, F. Pereira, K.Q. Weinberger (Eds.), Advances in Neural Information Processing Systems 24, NIPS Foundation Inc., Granada, 2011.

[12] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, T. Darrell, Caffe: Convolutional architecture for fast feature embedding, arXiv:1408.5093, 2014.

[13] TensorFlow: TensorFlow is an open source software library for machine intelligence. https://www.tensorflow.org/, 2015 (accessed 23.08.2016).

[14] V. Turchenko., A. Luczak, Creation of a deep convolutional auto-encoder in Caffe, arXiv:1512.01596, 2015. [OpenAIRE]

[15] M. Ranzato, F.J. Huang, Y.-L. Boureau, Y. LeCun, Unsupervised learning of invariant feature hierarchies with applications to object recognition, in: 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Minneapolis, MN, 2007, pp. 1-8.

28 references, page 1 of 2
Abstract
This paper presents the development of several models of a deep convolutional auto-encoder in the Caffe deep learning framework and their experimental evaluation on the example of MNIST dataset. We have created five models of a convolutional auto-encoder which differ architecturally by the presence or absence of pooling and unpooling layers in the auto-encoder's encoder and decoder parts. Our results show that the developed models provide very good results in dimensionality reduction and unsupervised clustering tasks, and small classification errors when we used the learned internal code as an input of a supervised linear classifier and multi-layer perceptron. T...
Subjects
ACM Computing Classification System: Data_CODINGANDINFORMATIONTHEORY
free text keywords: Computer Science - Neural and Evolutionary Computing, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Learning
Download from
28 references, page 1 of 2

[1] D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back-propagating errors, Nature. 323 (1986) 533-536. [OpenAIRE]

[2] Y. LeCun, Modeles connexionistes de l'apprentissage, Ph.D. thesis, Universite de Paris VI, 1987.

[3] H. Bourland, Y. Kamp, Auto-association by multilayer perceptrons and singular value decomposition, Biological Cybernetics. 59 (1988) 291-294. [OpenAIRE]

[4] P. Baldi, K. Hornik, Neural networks and principal component analysis: Learning from examples without local minima, Neural Networks. 2 (1989) 53-58. [OpenAIRE]

[5] G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science. 313 (2006) 504-507.

[6] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. of the IEEE. 86 (11) (1998) 2278-2324.

[7] Cuda-convnet2, High-performance C++/CUDA implementation of convolutional neural networks. https://github.com/akrizhevsky/cuda-convnet2, 2014 (accessed 23.08.2016).

[8] Theano Development Team, Theano: A Python framework for fast computation of mathematical expressions, arXiv: 1605.02688, 2016.

[9] Lasagne, Lasagne is a lightweight library to build and train neural networks in Theano. http://lasagne.readthedocs.org/, 2014 (accessed 23.08.2016).

[10] Keras: Deep learning library for Theano and TensorFlow. https://keras.io/, 2015 (accessed 23.08.2016).

[11] R. Collobert, K. Kavukcuoglu, C. Farabet, Torch7: A Matlab-like environment for machine learning, in: J. Shawe-Taylor, R.S. Zemel, P.L. Bartlett, F. Pereira, K.Q. Weinberger (Eds.), Advances in Neural Information Processing Systems 24, NIPS Foundation Inc., Granada, 2011.

[12] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, T. Darrell, Caffe: Convolutional architecture for fast feature embedding, arXiv:1408.5093, 2014.

[13] TensorFlow: TensorFlow is an open source software library for machine intelligence. https://www.tensorflow.org/, 2015 (accessed 23.08.2016).

[14] V. Turchenko., A. Luczak, Creation of a deep convolutional auto-encoder in Caffe, arXiv:1512.01596, 2015. [OpenAIRE]

[15] M. Ranzato, F.J. Huang, Y.-L. Boureau, Y. LeCun, Unsupervised learning of invariant feature hierarchies with applications to object recognition, in: 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Minneapolis, MN, 2007, pp. 1-8.

28 references, page 1 of 2
Powered by OpenAIRE Research Graph
Any information missing or wrong?Report an Issue