We investigated the representation of azimuthal directions of sound sources under two different conditions. In the first experiment, we examined the participants' mental representation of sound source directions via similarity judgments. Auditory stimuli originating from sixteen loudspeakers positioned equidistantly around the participant were presented in pairs, with the first stimulus serving as the anchor, and thereby providing the context for the second stimulus. For each pair of stimuli, participants had to rate the sound source directions as either similar or dissimilar. In the second experiment, the same participants categorized single sound source directions using verbal direction labels (front, back, left, right, and combinations of any two of these). In both experiments, the directions within the front and back regions were more distinctively categorized than those on the sides, and the sides' categories included more directions than those of the front or back. Furthermore, we found evidence that the left-right decision comprises the basic differentiation of the surrounding regions. These findings illustrate what seem to be central features of the representation of directions in auditory space.