In this paper, we start from an extension of the notion of holonomy on diffeological bundles, reformulate the notion of regular Lie group or Frölicher Lie groups, state an Ambrose–Singer theorem that enlarges the one stated in [J.-P. Magnot, Structure groups and holonomy in infinite dimensions, Bull. Sci. Math.128 (2004) 513–529], and conclude with a differential geometric treatment of KP hierarchy. The examples of Lie groups that are studied are principally those obtained by enlarging some graded Frölicher (Lie) algebras such as formal q-series of the quantum algebra of pseudo-differential operators. These deformations can be defined for classical pseudo-differential operators but they are used here on formal pseudo-differential operators in order to get a differential geometric framework to deal with the KP hierarchy that is known to be completely integrable with formal power series. Here, we get an integration of the Zakharov–Shabat connection form by means of smooth sections of a (differential geometric) bundle with structure group, some groups of q-deformed operators. The integration obtained by Mulase [Complete integrability of the Kadomtsev–Petviashvili equation Adv. Math.54 (1984) 57–66], and the key tools he developed, are totally recovered on the germs of the smooth maps of our construction. The tool coming from (classical) differential geometry used in this construction is the holonomy group, on which we have an Ambrose–Singer-like theorem: the Lie algebra is spanned by the curvature elements. This result is proved for any connection a diffeological principal bundle with structure group a regular Frölicher Lie group. The case of a (classical) Lie group modeled on a complete locally convex topological vector space is also recovered and the work developed in [J.-P. Magnot, Difféologie du fibré d'Holonomie en dimension infinie, Math. Rep. Canadian Roy. Math. Soc.28(4) (2006); J.-P. Magnot, Structure groups and holonomy in infinite dimensions, Bull. Sci. Math. 128 (2004) 513–529] is completed.