Learning Machines Implemented on Non-Deterministic Hardware

Preprint English OPEN
Gupta, Suyog ; Sindhwani, Vikas ; Gopalakrishnan, Kailash (2014)
  • Subject: Statistics - Machine Learning | Computer Science - Learning

This paper highlights new opportunities for designing large-scale machine learning systems as a consequence of blurring traditional boundaries that have allowed algorithm designers and application-level practitioners to stay -- for the most part -- oblivious to the details of the underlying hardware-level implementations. The hardware/software co-design methodology advocated here hinges on the deployment of compute-intensive machine learning kernels onto compute platforms that trade-off determinism in the computation for improvement in speed and/or energy efficiency. To achieve this, we revisit digital stochastic circuits for approximating matrix computations that are ubiquitous in machine learning algorithms. Theoretical and empirical evaluation is undertaken to assess the impact of the hardware-induced computational noise on algorithm performance. As a proof-of-concept, a stochastic hardware simulator is employed for training deep neural networks for image recognition problems.
  • References (23)
    23 references, page 1 of 3

    [1] LeĀ“on Bottou and Olivier Bousquet. The tradeoffs of large scale learning. In NIPS, volume 4, page 2, 2007.

    [2] H. Kushner and G.G. Lin. Springer, 2003.

    [3] M. Mahoney. Randomized algorithms for matrices and data. In Foundations and Trends in Machine Learning, 2011.

    [4] V. Chandrasekaran and M. I. Jordan. Computational and statistical tradeoffs via convex relaxation. In Proceedings of the National Academy of Sciences, 2013.

    [5] WJ Poppelbaum, C Afuso, and JW Esch. Stochastic computing elements and systems. In Proceedings of the November 14-16, 1967, fall joint computer conference, pages 635-644. ACM, 1967.

    [6] BR Gaines. Stochastic computing systems. In Advances in information systems science, pages 37-172. Springer, 1969.

    [7] Armin Alaghi and John P Hayes. Survey of stochastic computing. ACM Transactions on Embedded computing systems (TECS), 12(2s):92, 2013.

    [8] Lifeng Miao and Chaitali Chakrabarti. A parallel stochastic computing system with improved accuracy. In Signal Processing Systems (SiPS), 2013 IEEE Workshop on, pages 195-200. IEEE, 2013.

    [9] P Knag, W Lu, and Z Zhang. A native stochastic computing architecture enabled by memristors. Nanotechnology, IEEE Transactions on, 13(2):283-293, 2014.

    [10] Armin Alaghi and John P Hayes. Fast and accurate computation using stochastic circuits. In Proceedings of the conference on Design, Automation & Test in Europe, page 76. European Design and Automation Association, 2014.

  • Metrics
    No metrics available
Share - Bookmark