Let $G$ be a group acting faithfully on a set $X$. The distinguishing number of the action of $G$ on $X$, denoted $D_G(X)$, is the smallest number of colors such that there exists a coloring of $X$ where no nontrivial group element induces a color-preserving permutation of $X$. In this paper, we show that if $G$ is nilpotent of class $c$ or supersolvable of length $c$ then $G$ always acts with distinguishing number at most $c+1$. We obtain that all metacyclic groups act with distinguishing number at most 3; these include all groups of squarefree order. We also prove that the distinguishing number of the action of the general linear group $GL_n(K)$ over a field $K$ on the vector space $K^n$ is 2 if $K$ has at least $n+1$ elements.