In this paper, we are concerned with finding exact solutions for the stress fields of nonlinear solids with non-symmetric distributions of defects (or more generally finite eigenstrains) that are small perturbations of symmetric distributions of defects with known exact solutions. In the language of geometric mechanics, this corresponds to finding a deformation that is a result of a perturbation of the metric of the Riemannian material manifold. We present a general framework that can be used for a systematic analysis of this class of anelasticity problems. This geometric formulation can be thought of as a material analogue of the classical small-on-large theory in nonlinear elasticity. We use the present small-on-large anelasticity theory to find exact solutions for the stress fields of some non-symmetric distributions of screw dislocations in incompressible isotropic solids.