Although knowledge of the selective agents behind the evolution of sexual dimorphism has advanced considerably in recent years, we still lack a clear understanding of the evolutionary durability of cross-sex genetic covariances that often constrain its evolution. We tested the relative stability of cross-sex genetic covariances for a suite of homologous contact pheromones of the fruit fly Drosophila serrata, along a latitudinal gradient where these traits have diverged in mean. Using a Bayesian framework, which allowed us to account for uncertainty in all parameter estimates, we compared divergence in the total amount and orientation of genetic variance across populations, finding divergence in orientation but not total variance. We then statistically compared orientation divergence of within-sex (G) to cross-sex (B) covariance matrices. In line with a previous theoretical prediction, we find that the cross-sex covariance matrix, B, is more variable than either within-sex G matrix. Decomposition of B matrices into their symmetrical and nonsymmetrical components revealed that instability is linked to the degree of asymmetry. We also find that the degree of asymmetry correlates with latitude suggesting a role for spatially varying natural selection in shaping genetic constraints on the evolution of sexual dimorphism.