In this paper we propose a new multivariate GARCH model with time-varying correlations. We adopt the vech representation based on the conditional variances and the conditional correlations. While each conditional-variance term is assumed to follow a univariate GARCH formulation, the conditional-correlation matrix is postulated to follow an autoregressive moving average type of analogue. By imposing some suitable restrictions on the conditional-correlation-matrix equation, we construct a MGARCH model in which the conditional-correlation matrix is guaranteed to be positive definite during the optimisation. Thus, our new model retains the intuition and interpretation of the univariate GARCH model and yet satisfies the positive-definite condition as found in the constant-correlation and BEKK models. We report some Monte Carlo results on the finite-sample distributions of the MLE of the varying-correlation MGARCH model. The new model is applied to some real data sets. It is found that extending the constant-correlation model to allow for time-varying correlations provides some interesting time histories that are not available in a constant-correlation model.