Continuing the considerations from the paper ``Bundles, linear bundles and principal bundles in the category of differential spaces'' [to appear ibid. 26 (1993)] the author carries over here to the theory of differential spaces the notions of connection and curvature and certain fundamental propositions from the theory of differential manifolds. So, the proposition giving the geometrical interpretation of the Lie bracket of vector fields on differential manifolds is generalized to the case of differential spaces. Moreover, the author proves that the structure equation of Maurer-Cartan and the Bianchi identity are fulfilled on an arbitrary principal bundle of a differential space with connection.