Gauge-invariant operators can be specified by equivalence classes of permutations. We develop this idea concretely for the singlets of the flavour group $SO(N_f)$ in $U(N_c)$ gauge theory by using Gelfand pairs and Schur-Weyl duality. The singlet operators, when specialised at $N_f =6$, belong to the scalar sector of ${\cal N}=4$ SYM. A simple formula is given for the two-point functions in the free field limit of $g_{YM}^2 =0$. The free two-point functions are shown to be equal to the partition function on a 2-complex with boundaries and a defect, in a topological field theory of permutations. The permutation equivalence classes are Fourier transformed to a representation basis which is orthogonal for the two-point functions at finite $N_c , N_f$. Counting formulae for the gauge-invariant operators are described. The one-loop mixing matrix is derived as a linear operator on the permutation equivalence classes.

50 pages, v2: typos corrected, v3: to appear in JHEP