We are at a crossroads in the study of Heliophysics. On one hand we operate in the same paradigm that has guided the field over the past couple of decades, ruled by the triumvirate of data, theory, and simulations. On the other hand, we are beginning to recognize that powerful new opportunities for scientific discovery are possible through increased data volume and sophisticated methods to explore these data. This paper focuses on an important and poorly communicated side of data science and machine learning: the application of these methods as discovery tools. We cover the progress of data science and machine learning in Heliophysics at a high level and establish the importance of the notion of explainability of models.