Abstract Studies of learning mechanisms critically depend on the ability to accurately assess learning outcomes. This assessment can be impeded by the often complex, multidimensional nature of behavior. We present a novel, automated approach to evaluating imitative learning that is founded in information theory. Conceptually, our approach estimates the amount of information present in a reference behavior that is absent from the learned behavior. We validate our approach through examination of songbird vocalizations, complex learned behaviors the study of which has provided many insights into sensory-motor learning in general and vocal learning in particular. Historically, learning has been holistically assessed by human inspection or through comparison of specific song features selected by experimenters (e.g. fundamental frequency, spectral entropy). In contrast, our approach relies on statistical models that broadly capture the structure of each song, and then uses these models to estimate the amount of information in the reference song but absent from the learned song. We show that our information theoretic measure of song learning (contrast entropy) is well correlated with human evaluation of song learning. We then expand the analysis beyond song learning and show that contrast entropy also detects the typical song deterioration that occurs following deafening. More broadly, this approach potentially provides a framework for assessing learning across a broad range of similarly structured behaviors. Author Summary Measuring learning outcomes is a critical objective of research into the neural, molecular, and behavioral mechanisms that support learning. Demonstration that a given manipulation results in better or worse learning outcomes requires an accurate and consistent measurement of learning quality. However, many behaviors (e.g. speech, walking, and reading) are complex and multidimensional, confounding the assessment of learning. One behavior subject to such confounds, vocal learning in Estrildid finches, has emerged as a vital model for sensory motor learning broadly and human speech learning in particular. Here, we demonstrate a new approach, founded in information theory, to the assessment of learning for complex high dimensional behaviors. Conceptually, we determine the amount of information (across many dimensions) present in a reference behavior and then assess how much of that information is present in the resultant learned behavior. We show that this measure provides an accurate, holistic, and automated assessment of vocal learning in Estrildid finches. Potentially, this same approach could be deployed to assess shared content in any multidimensional data, behavioral or otherwise.