When presented with the power of modern machine learning techniques, there is a belief that we can simply let these algorithms loose on the data and see what they can find, unconstrained by human choice or bias. While such approaches can be useful, they are (of course) not fully free of bias or choice. Moreover, by utilizing the deep store of knowledge built up by scientific domains over decades or centuries, we can make architectural choices in our machine learning algorithms that focus the learning on features that we already know are important and informative, leading to more efficient, explainable, and interpretable methods. In this work, we present three examples of this approach. In the first project, to make use of the knowledge that texture is an important attribute of clouds, we use tools from topological data analysis focusing on the texture of satellite imagery, which leads to an effective and highly interpretable classifier of mesoscale cloud organization. This project resulted in a paper that has been published as a journal article. In the second project, we compare a rotationally invariant convolutional neural network against a conventional CNN both with and without data augmentation in their performance and behaviors on the task of predicting the major and minor axes lengths of storms in forecast data. Finally, in the third project, we explore three different techniques from harmonic analysis to enhance the signature of gravity waves in satellite imagery.