We show that the spin factors that appear in the partition function for spinning particles are of geometrical origin and can be expressed as Wilson loops over gauge potentials that can be calculated from the tangent vector of the curve. The spin factor can be explicitly evaluated in two and three dimensions, while in four dimensions it is expressed as a Wilson loop over an explicitly given SU(2) gauge potential.