We investigate relationship between a gauge theory on a principal bundle and that on its base space. In the case where the principal bundle is itself a group manifold, we also study relations of those gauge theories with a matrix model obtained by dimensionally reducing them to zero dimensions. First, we develop the dimensional reduction of Yang-Mills (YM) on the total space to YM-higgs on the base space for a general principal bundle. Second, we show a relationship that YM on an SU(2) bundle is equivalent to the theory around a certain background of YM-higgs on its base space. This is an extension of our previous work (hep-th/0703021), in which the same relationship concerning a U(1) bundle is shown. We apply these results to the case of $SU(n+1)$ as the total space. By dimensionally reducing YM on $SU(n+1)$, we obtain YM-higgs on $SU(n+1)/SU(n)\simeq S^{2n+1}$ and on $SU(n+1)/(SU(n)\times U(1))\simeq CP^n$ and a matrix model. We show that the theory around each monopole vacuum of YM-higgs on $CP^n$ is equivalent to the theory around a certain vacuum of the matrix model in the commutative limit. By combing this with the relationship concerning a U(1) bundle, we realize YM-higgs on $SU(n+1)/SU(n)\simeq S^{2n+1}$ in the matrix model. We see that the relationship concerning a U(1) bundle can be interpreted as Buscher's T-duality.

53 pages, references added, typos corrected