Various new generalized forms of the Gegenbauer matrix polynomials are introduced using the integral representation method, which allows us to express them in terms of Hermite matrix polynomials. Certain properties for these new generalized Gegenbauer matrix polynomials such as recurrence relations and expansion in terms of Hermite matrix polynomials are derived. Further, several families of bilinear and bilateral generating matrix relations for these polynomials are established and their applications are presented.