In this paper, we explicitly provide expressions for a sequence of orthogonal polynomials associated with a weight matrix of size N , constructed from a collection of scalar weights w_{1}, \ldots, w_{N} of the form W(x) = T(x)\operatorname{diag}(w_{1}(x), \ldots, w_{N}(x))T(x)^{\ast} , where T(x) is a specific polynomial matrix. We provide sufficient conditions on the scalar weights to ensure that the weight matrix W is irreducible. Furthermore, we give sufficient conditions on the scalar weights to ensure that each term in the constructed sequence of matrix orthogonal polynomials is an eigenfunction of a differential operator. We also study the Darboux transformations and bispectrality of the orthogonal polynomials in the particular case where the scalar weights are the classical weights of Jacobi, Hermite, and Laguerre. With these results, we construct a wide variety of bispectral matrix-valued orthogonal polynomials of arbitrary size, which satisfy a second-order differential equation.