Given a possibly reducible and non-reduced spectral cover X over a smooth projective complex curve C we determine the group of connected components of the Prym variety Prym(X/C). As an immediate application we show that the finite group of n-torsion points of the Jacobian of C acts trivially on the cohomology of the twisted SL_n-Higgs moduli space up to the degree which is predicted by topological mirror symmetry. In particular this yields a new proof of a result of Harder--Narasimhan, showing that this finite group acts trivially on the cohomology of the twisted SL_n stable bundle moduli space.

22 pages, applications to topological mirror symmetry and a result of Harder--Narasimhan are added