We calculate the autocorrelation functions (or shifted moments) of the characteristic polynomials of matrices drawn uniformly with respect to Haar measure from the groups U(N), O(2N) and USp(2N). In each case the result can be expressed in three equivalent forms: as a determinant sum (and hence in terms of symmetric polynomials), as a combinatorial sum, and as a multiple contour integral. These formulae are analogous to those previously obtained for the Gaussian ensembles of Random Matrix Theory, but in this case are identities for any size of matrix, rather than large-matrix asymptotic approximations. They also mirror exactly autocorrelation formulae conjectured to hold for L-functions in a companion paper. This then provides further evidence in support of the connection between Random Matrix Theory and the theory of L-functions.