Dipolar interactions are crucial in the modeling of many complex magnetic systems, such as the pyrochlores and artificial spin systems. Remarkably, many classical dipolar coupled spin systems exhibit a continuous ground-state degeneracy, which is unexpected as the Hamiltonian does not possess a continuous symmetry. In this paper, we explain, how such a finite point-symmetry leads to a continuous ground-state degeneracy of specific classical dipolar-coupled systems. This work, therefore, provides new insight into the theory of classical dipolar-coupled spin-systems and opens the way to understand more complex dipolar coupled systems.