We study the issue of diffeomorphism symmetry in group field theories (GFT), using the recently introduced noncommutative metric representation. In the colored Boulatov model for 3d gravity, we identify a field (quantum) symmetry which ties together the vertex translation invariance of discrete gravity, the flatness constraint of canonical quantum gravity, and the topological (coarse-graining) identities for the 6j-symbols. We also show how, for the GFT graphs dual to manifolds, the invariance of the Feynman amplitudes encodes the discrete residual action of diffeomorphisms in simplicial gravity path integrals. We extend the results to GFT models for higher dimensional BF theories and discuss various insights that they provide on the GFT formalism itself.

31 pages, revtex4; many figures