publication . Preprint . 2016

Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks

Gysel, Philipp;
Open Access English
  • Published: 20 May 2016
Abstract
Convolutional neural networks (CNN) have achieved major breakthroughs in recent years. Their performance in computer vision have matched and in some areas even surpassed human capabilities. Deep neural networks can capture complex non-linear features; however this ability comes at the cost of high computational and memory requirements. State-of-art networks require billions of arithmetic operations and millions of parameters. To enable embedded devices such as smartphones, Google glasses and monitoring cameras with the astonishing power of deep learning, dedicated hardware accelerators can be used to decrease both execution time and power consumption. In applica...
Subjects
free text keywords: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Learning, Computer Science - Neural and Evolutionary Computing
Download from
37 references, page 1 of 3

2 Convolutional Neural Networks 3 2.1 Training and Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Layer Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Computational Complexity and Memory Requirements . . . . . . . . . . 11 2.5 ImageNet Competition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.6 Neural Networks With Limited Numerical Precision . . . . . . . . . . . . 14

3 Related Work 19 3.1 Network Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 Accelerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4 Fixed Point Approximation 25 4.1 Baseline Convolutional Neural Networks . . . . . . . . . . . . . . . . . . 25 4.2 Fixed Point Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.3 Dynamic Range of Parameters and Layer Outputs . . . . . . . . . . . . . 27 4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5 Dynamic Fixed Point Approximation 32 5.1 Mixed Precision Fixed Point . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.2 Dynamic Fixed Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

8 Comparison of Di erent Approximations 47 8.1 Fixed Point Approximation . . . . . . . . . . . . . . . . . . . . . . . . . 48 8.2 Dynamic Fixed Point Approximation . . . . . . . . . . . . . . . . . . . . 49 8.3 Mini oat Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 8.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

9 Ristretto: An Approximation Framework for Deep CNNs 51 9.1 From Ca e to Ristretto . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 9.2 Quantization Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 9.3 Fine-tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 9.4 Fast Forward and Backward Propagation . . . . . . . . . . . . . . . . . . 53 9.5 Ristretto From a User Perspective . . . . . . . . . . . . . . . . . . . . . . 54 9.6 Release of Ristretto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 9.7 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

4.1 Dynamic range of values in LeNet . . . . . . . . . . . . . . . . . . . . . . 27

4.2 Dynamic range of values in Ca eNet . . . . . . . . . . . . . . . . . . . . 28

4.3 Fixed point results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5.1 Fixed point data path . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5.2 Dynamic xed point representation . . . . . . . . . . . . . . . . . . . . . 34

5.3 Static vs dynamic xed point . . . . . . . . . . . . . . . . . . . . . . . . 35

6.1 Mini oat number representation . . . . . . . . . . . . . . . . . . . . . . . 39

6.2 Mini oat data path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

6.3 Mini oat results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

37 references, page 1 of 3
Abstract
Convolutional neural networks (CNN) have achieved major breakthroughs in recent years. Their performance in computer vision have matched and in some areas even surpassed human capabilities. Deep neural networks can capture complex non-linear features; however this ability comes at the cost of high computational and memory requirements. State-of-art networks require billions of arithmetic operations and millions of parameters. To enable embedded devices such as smartphones, Google glasses and monitoring cameras with the astonishing power of deep learning, dedicated hardware accelerators can be used to decrease both execution time and power consumption. In applica...
Subjects
free text keywords: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Learning, Computer Science - Neural and Evolutionary Computing
Download from
37 references, page 1 of 3

2 Convolutional Neural Networks 3 2.1 Training and Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Layer Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Computational Complexity and Memory Requirements . . . . . . . . . . 11 2.5 ImageNet Competition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.6 Neural Networks With Limited Numerical Precision . . . . . . . . . . . . 14

3 Related Work 19 3.1 Network Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 Accelerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4 Fixed Point Approximation 25 4.1 Baseline Convolutional Neural Networks . . . . . . . . . . . . . . . . . . 25 4.2 Fixed Point Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.3 Dynamic Range of Parameters and Layer Outputs . . . . . . . . . . . . . 27 4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5 Dynamic Fixed Point Approximation 32 5.1 Mixed Precision Fixed Point . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.2 Dynamic Fixed Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

8 Comparison of Di erent Approximations 47 8.1 Fixed Point Approximation . . . . . . . . . . . . . . . . . . . . . . . . . 48 8.2 Dynamic Fixed Point Approximation . . . . . . . . . . . . . . . . . . . . 49 8.3 Mini oat Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 8.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

9 Ristretto: An Approximation Framework for Deep CNNs 51 9.1 From Ca e to Ristretto . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 9.2 Quantization Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 9.3 Fine-tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 9.4 Fast Forward and Backward Propagation . . . . . . . . . . . . . . . . . . 53 9.5 Ristretto From a User Perspective . . . . . . . . . . . . . . . . . . . . . . 54 9.6 Release of Ristretto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 9.7 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

4.1 Dynamic range of values in LeNet . . . . . . . . . . . . . . . . . . . . . . 27

4.2 Dynamic range of values in Ca eNet . . . . . . . . . . . . . . . . . . . . 28

4.3 Fixed point results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5.1 Fixed point data path . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5.2 Dynamic xed point representation . . . . . . . . . . . . . . . . . . . . . 34

5.3 Static vs dynamic xed point . . . . . . . . . . . . . . . . . . . . . . . . 35

6.1 Mini oat number representation . . . . . . . . . . . . . . . . . . . . . . . 39

6.2 Mini oat data path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

6.3 Mini oat results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

37 references, page 1 of 3
Powered by OpenAIRE Research Graph
Any information missing or wrong?Report an Issue