The Gaussian Belief Propagation (GaBP) algorithm executed on Gaussian Markov Random Fields can take a large number of iterations to converge if the inverse covariance matrix of the underlying Gaussian distribution is ill-conditioned and weakly diagonally dominant. Such matrices can arise from many practical problem domains. In this study, we propose a relaxed GaBP algorithm that results in a significant reduction in the number of GaBP iterations (of up to 12.7 times). We also propose a second relaxed GaBP algorithm that avoids the need of determining the relaxation factor a priori which can also achieve comparable reductions in iterations by only setting two basic heuristic measures. We show that the new algorithms can be implemented without any significant increase, over the original GaBP, in both the computational complexity and the memory requirements. We also present detailed experimental results of the new algorithms and demonstrate their effectiveness in achieving significant reductions in the iteration count.