The boundary of a boundary principle in field theories is described. The difference in treatment of the principle in electrodynamics and general relativity is pointed out and reformulated in terms of underlying mathematical structure of the theories. The problem of unifying the treatment is formulated and solved. The role of E. Cartan's concept of the moment of rotation associated with the curvature of a Levi-Civita connection on a frame bundle is shown to be crucial for the unification. The analysis of the boundary of a boundary principle in Kaluza-Klein theory is performed and the recipe for a unified treatment of the principle in electrodynamics and general relativity is shown to follow from the analysis. It is pointed out that the unification cand be extended to Yang-Mills fields easily.