Let \(H_n\) be the space of \(n\times n\) Hermitian matrices and let \(D_n\cong \mathbb R^n\) be the subspace of real diagonal matrices. A central function \(\hat{P}\) on \(H_n\) so that its restriction to the diagonal matrices is a symmetric polynomial \(P\) on \(\mathbb R^n\) is called a (generalized) polynomial of Hermitian matrix argument. Let \(\mu\) be a finite Borel measure on \(\mathbb R\) and let \(V(x_1,\dots,x_n)\) denote the Vandermonde determinant. Moreover, let \(M\) be the central measure on \(H_n\) which agrees on \(D_n\cong \mathbb R^n\) with the permutation invariant measure \(\mu_n=V^2(x_1,\dots,x_n) d\mu(x_1) \cdots d\mu(x_n)\). This paper constructs and studies the basis of generalized polynomials of Hermitian matrix argument which are orthogonal with respect to \(M\). Under some assumptions on the measure \(\mu\), the authors associate with an orthogonal family of polynomials in \(L^2(\mathbb R,\mu)\), a family of symmetric polynomials \(P_\lambda\) by means of a Berezin-Karpelevic type formula. The \(P_\lambda\), which are parameterized by partitions \(\lambda\) of length smaller or equal to \(n\), form an orthogonal Hilbert basis for the space of symmetric polynomials in \(L^2(\mathbb R^n,\mu_n)\). They can also be obtained by the Gram-Schmidt orthogonalization process applied to the family of Schur polynomials ordered in the graded lexicographic order. The corresponding generalized polynomials of Hermitian matrix argument \(\hat{P}_\lambda\) form an orthogonal Hilbert basis for the space of central functions on \(H_n\) which are \(L^2\) with respect to \(M\). Special cases of this general construction are the generalized Hermite, Jacobi and Laguerre polynomials, which have been studied by \textit{M. Lassalle} in a series of notes [C. R. Acad. Sci., Paris, Sér I 312, No. 6, 425--428 (1991; Zbl 0742.33005); ibid. 313, No. 9, 579--582 (1991; Zbl 0748.33006); ibid. 312, No. 10, 725--728 (1991; Zbl 0739.33007)].