Cross-sectional imaging techniques—echocardiography, CT, MRI and nuclear medicine—are the diagnostic tools of choice for the diagnosis and workup of cardiovascular disease. Machine learning and deep learning in particular will have a fundamental and lasting impact on all of these modalities. Whereas deep learning is mostly discussed in the context of image interpretation, we show that the impact is much broader than this. The entire imaging chain from choosing the appropriate imaging test to acquiring the proper images, reconstruction of images from raw data, image interpretation, reporting and derivation of prognostic information can be improved by application of machine learning and deep learning techniques. Application of machine learning and deep learning algorithms will be an important step towards fulfilling the promise of truly personalized medicine, especially when information from imaging is combined with other data such as the results from laboratory evaluations, genetic analysis, medication use and personal fitness trackers. Nevertheless, the process of bringing the results to physicians is nontrivial, and we also discuss our experience with deployment of developed algorithms in clinical practice.