publication . Preprint . 2015

Reducing the Training Time of Neural Networks by Partitioning

Miranda, Conrado S.; Von Zuben, Fernando J.;
Open Access English
  • Published: 09 Nov 2015
Abstract
Comment: Figure 2b has lower quality due to file size constraints
Subjects
free text keywords: Computer Science - Neural and Evolutionary Computing, Computer Science - Learning
Download from
24 references, page 1 of 2

Bengio, Y., Goodfellow, I. J., and Courville, A. Deep learning. Book in preparation for MIT Press, 2015. URL www.iro.umontreal.ca/~bengioy/dlbook.

Chen, T., Goodfellow, I., and Shlens, J. arXiv:1511.05641, 2015.

Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J., Catanzaro, B., and Shelhamer, E. cuDNN: Efficient primitives for deep learning. arXiv:1410.0759, 2014. [OpenAIRE]

Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B., and LeCun, Y. The loss surface of multilayer networks. arXiv:1412.0233, 2014. [OpenAIRE]

Coates, A., Huval, B., Wang, T., Wu, D. J., Ng, A. Y., and Catanzaro, B. Deep learning with COTS HPC systems. In Proceedings of the 30th International Conference on Machine Learning, pp. 1337-1345, 2013.

Dauphin, Y. N., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., and Bengio, Y. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In Advances in Neural Information Processing Systems, pp. 2933-2941, 2014. [OpenAIRE]

Dean, J., Corrado, G. S., Monga, R., Chen, K., Devin, M., Le, Q. V., Mao, M. Z., Ranzato, M., Senior, A., Tucker, P., Yang, K., and Ng, A. Y. Large scale distributed deep networks. In Advances in Neural Information Processing Systems, pp. 1223-1231, 2012.

Erhan, D., Bengio, Y., Courville, A., Manzagol, P., Vincent, P., and Bengio, S. Why does unsupervised pretraining help deep learning? The Journal of Machine Learning Research, 11:625-660, 2010. [OpenAIRE]

Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In International Conference on Artificial Intelligence and Statistics, pp. 249-256, 2010.

Gupta, S., Agrawal, A., Gopalakrishnan, K., and Narayanan, P. Deep learning with limited numerical precision. arXiv:1502.02551, 2015.

Hansen, L. K. and Salamon, P. Neural network ensembles. IEEE Transactions on Pattern Analysis & Machine Intelligence, (10):993-1001, 1990.

He, K., Zhang, X., Ren, S., and Sun, J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv:1502.01852, 2015.

Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. arXiv:1503.02531, 2015.

Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, 2009.

Krizhevsky, A. One weird trick for parallelizing convolutional neural networks. arXiv:1404.5997, 2014. [OpenAIRE]

24 references, page 1 of 2
Abstract
Comment: Figure 2b has lower quality due to file size constraints
Subjects
free text keywords: Computer Science - Neural and Evolutionary Computing, Computer Science - Learning
Download from
24 references, page 1 of 2

Bengio, Y., Goodfellow, I. J., and Courville, A. Deep learning. Book in preparation for MIT Press, 2015. URL www.iro.umontreal.ca/~bengioy/dlbook.

Chen, T., Goodfellow, I., and Shlens, J. arXiv:1511.05641, 2015.

Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J., Catanzaro, B., and Shelhamer, E. cuDNN: Efficient primitives for deep learning. arXiv:1410.0759, 2014. [OpenAIRE]

Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B., and LeCun, Y. The loss surface of multilayer networks. arXiv:1412.0233, 2014. [OpenAIRE]

Coates, A., Huval, B., Wang, T., Wu, D. J., Ng, A. Y., and Catanzaro, B. Deep learning with COTS HPC systems. In Proceedings of the 30th International Conference on Machine Learning, pp. 1337-1345, 2013.

Dauphin, Y. N., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., and Bengio, Y. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In Advances in Neural Information Processing Systems, pp. 2933-2941, 2014. [OpenAIRE]

Dean, J., Corrado, G. S., Monga, R., Chen, K., Devin, M., Le, Q. V., Mao, M. Z., Ranzato, M., Senior, A., Tucker, P., Yang, K., and Ng, A. Y. Large scale distributed deep networks. In Advances in Neural Information Processing Systems, pp. 1223-1231, 2012.

Erhan, D., Bengio, Y., Courville, A., Manzagol, P., Vincent, P., and Bengio, S. Why does unsupervised pretraining help deep learning? The Journal of Machine Learning Research, 11:625-660, 2010. [OpenAIRE]

Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In International Conference on Artificial Intelligence and Statistics, pp. 249-256, 2010.

Gupta, S., Agrawal, A., Gopalakrishnan, K., and Narayanan, P. Deep learning with limited numerical precision. arXiv:1502.02551, 2015.

Hansen, L. K. and Salamon, P. Neural network ensembles. IEEE Transactions on Pattern Analysis & Machine Intelligence, (10):993-1001, 1990.

He, K., Zhang, X., Ren, S., and Sun, J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv:1502.01852, 2015.

Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. arXiv:1503.02531, 2015.

Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, 2009.

Krizhevsky, A. One weird trick for parallelizing convolutional neural networks. arXiv:1404.5997, 2014. [OpenAIRE]

24 references, page 1 of 2
Powered by OpenAIRE Research Graph
Any information missing or wrong?Report an Issue