Before investing in long-term monitoring or reinforcement of structures, it is essential to understand underlying damage mechanisms and consequences for structural stability. Approaches combining nondestructive evaluation and finite element modeling have been successful in producing qualitative diagnoses for damage to existing structures. However, the real-world impact of such methods will hinge upon a reduced computational burden and improved accuracy of comparison between models and physical infrastructure. This chapter describes a new approach based on unsupervised learning to perform quantitative damage state inversion from sparse datasets. Discrete element modeling was used to simulate the response of masonry walls and other structures under settlement loading. Point cloud representations of the structures, consistent with modern computer vision pipelines used for documentation, were used to generate a low-dimensional manifold based on the Wasserstein metric. This manifold is used to train a Gaussian process model which can then be interrogated to infer loading conditions from the damage state. This method is shown to quantitatively reproduce the loading conditions for masonry structures and was validated against laboratory-scale, experimental masonry walls. Although the approach is demonstrated here for settlement-induced cracking, it has important implications for the broader field of data-driven diagnostics for discontinuous media.