publication . Preprint . 2014

Building Program Vector Representations for Deep Learning

Mou, Lili; Li, Ge; Liu, Yuxuan; Peng, Hao; Jin, Zhi; Xu, Yan; Zhang, Lu;
Open Access English
  • Published: 11 Sep 2014
Abstract
Deep learning has made significant breakthroughs in various fields of artificial intelligence. Advantages of deep learning include the ability to capture highly complicated features, weak involvement of human engineering, etc. However, it is still virtually impossible to use deep learning to analyze programs since deep architectures cannot be trained effectively with pure back propagation. In this pioneering paper, we propose the "coding criterion" to build program vector representations, which are the premise of deep learning for program analysis. Our representation learning approach directly makes deep learning a reality in this new field. We evaluate the lear...
Subjects
free text keywords: Computer Science - Software Engineering, Computer Science - Learning, Computer Science - Neural and Evolutionary Computing
Download from
58 references, page 1 of 4

[1] H. Lu, B. Cukic, and M. Culp, “Software defect prediction using semi-supervised learning with dimension reduction,” in Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, 2012. [OpenAIRE]

[2] S. Lee, C. Jung, and S. Pande, “Detecting memory leaks through introspective dynamic behavior modelling using machine learning,” in Proceedings of 36th International Conference on Software Engineering, 2014.

[3] K. Canavera, N. Esfahani, and S. Malek, “Mining the execution history of a software system to infer the best time for its adaptation,” in Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, 2012.

[4] A. Hindle, E. Barr, Z. Su, M. Gabel, and P. Devanbu, “On the naturalness of software,” in Proceedings of 34th International Conference on Software Engineering, 2012.

[5] G. Hinton, S. Osindero, and Y. Teh, “A fast learning algorithm for deep belief nets,” Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006.

[6] R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa, “Natural language processing (almost) from scratch,” The Journal of Machine Learning Research, vol. 12, pp. 2493-2537, 2011. [OpenAIRE]

[7] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. Manning, A. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank,” in Proceedings of Conference on Empirical Methods in Natural Language Processing, 2013.

[8] A. Krizhevsky, I. Sutskever, and G. Hinton, “ImageNet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems, 2012.

[9] D. Ciresan, U. Meier, and J. Schmidhuber, “Multi-column deep neural networks for image classification,” in IEEE Conference on Computer Vision and Pattern Recognition, 2012. [OpenAIRE]

[10] G. Dahl, A. Mohamed, and G. E. Hinton, “Phone recognition with the mean-covariance restricted Boltzmann machine,” in Advances in Neural Information Processing Systems, 2010.

[11] A. Mohamed, G. Dahl, and G. Hinton, “Acoustic modeling using deep belief networks,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 20, no. 1, pp. 14-22, 2012.

[12] R. Collobert and J. Weston, “A unified architecture for natural language processing: Deep neural networks with multitask learning,” in Proceedings of the 25th International Conference on Machine learning, 2008. [OpenAIRE]

[13] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layerwise training of deep networks,” in Advances in Neural Information Processing Systems, 2007.

[14] D. Erhan, P. Manzagol, Y. Bengio, S. Bengio, and P. Vincent, “The difficulty of training deep architectures and the effect of unsupervised pre-training,” in Proceedings of International Conference on Artificial Intelligence and Statistics, 2009.

[15] H. Larochelle, Y. Bengio, J. Louradour, and P. Lamblin, “Exploring strategies for training deep neural networks,” The Journal of Machine Learning Research, vol. 10, pp. 1-40, 2009.

58 references, page 1 of 4
Any information missing or wrong?Report an Issue