Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks

Preprint English OPEN
Shen, Li; Lin, Zhouchen; Huang, Qingming;
  • Subject: Computer Science - Computer Vision and Pattern Recognition | Computer Science - Learning

Learning deeper convolutional neural networks becomes a tendency in recent years. However, many empirical evidences suggest that performance improvement cannot be gained by simply stacking more layers. In this paper, we consider the issue from an information theoretical... View more
  • References (29)
    29 references, page 1 of 3

    1. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classi cation with deep convolutional neural networks. In: NIPS. (2012)

    2. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR. (2014)

    3. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap to human-level performance in face veri cation. In: CVPR. (2014)

    4. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Largescale video classi cation with convolutional neural networks. In: CVPR. (2014)

    5. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR. (2015)

    6. Srivastava, R.K., Gre , K., Schmidhuber, J.: Highway networks. In: ICML Deep Learning Workshop. (2015)

    7. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into recti ers: Surpassing humanlevel performance on imagenet classi cation. In: ICCV. (2015)

    8. Zhou, B., Khosla, A., Lapedriza, A., Torralba, A., Oliva, A.: Places2: A large-scale database for scene understanding. (2015)

    9. LeCun, Y., Bottou, L., Orr, G., Muller, K.: E cient backprop. Neural Networks: Tricks of the trade (1998)

    10. Nair, V., Hinton, G.: Recti ed linear units improve restricted boltzmann machines. In: ICML. (2010)

  • Metrics
Share - Bookmark