The basic rules of tensor analysis in non-Euclidean spaces are derived by means of the formalism of commutative diagrams (widely used in many branches of mathematics, especially the theory of categories). We consider here as an example the case of general relativity (although this approach can be applied to gauge theories as well). The different dimensionality of the diagrams involved gives rise naturally to a hierarchy of the corresponding physical relations, starting from the simplest differential object—covariant derivative—to Bianchi identities and Einstein’s equations. The commutative-diagram approach allows one to single out in a natural way three basic postulates, which can be applied to build up any gauge theory.