With Deep Learning becoming ubiquitous in our life, running Deep Learning algorithms in real time on an heterogeneous set of hardware platforms is a pressing need in many aspects of our society. While traditional workflows based on standard CPUs and GPUs are established, Deep Learning inference on low-power devices (e.g., cars, smart phones, watches, etc) is gaining more attention. Typically, this would require strong background in electronic engineering to convert a neural network into a Digital Signal Processor. We propose to develop a complete open-software library to automatically convert Deep Neural Networks to electronic circuits, using High Level Synthesis tools. With a large basis of potential applications (e.g., autonomous cars, medical devices, portable monitoring devices, custom electronics as in the real-time data processing system of large-scale scientific experiments, etc.), the hls4ml library would assists users by automatising the logic circuit design as well as by reducing resource utilisation while preserving accuracy.