Courtship behavior in Drosophila has been causally linked to the activity of the heterogeneous set of ∼1500 neurons that express the sex-specific transcripts of the fruitless (fru) gene, but we currently lack an appreciation of the cellular diversity within this population, the extent to which these cells are sexually dimorphic, and how they might be organized into functional circuits.We used genetic methods to define 100 distinct classes of fru neuron, which we compiled into a digital 3D atlas at cellular resolution. We determined the polarity of many of these neurons and computed their likely patterns of connectivity, thereby assembling them into a neural circuit that extends from sensory input to motor output. The cellular organization of this circuit reveals neuronal pathways in the brain that are likely to integrate multiple sensory cues from other flies and to issue descending control signals to motor circuits in the thoracic ganglia. We identified 11 anatomical dimorphisms within this circuit: neurons that are male specific, are more numerous in males than females, or have distinct arborization patterns in males and females.The cellular organization of the fru circuit suggests how multiple distinct sensory cues are integrated in the fly's brain to drive sex-specific courtship behavior. We propose that sensory processing and motor control are mediated through circuits that are largely similar in males and females. Sex-specific behavior may instead arise through dimorphic circuits in the brain and nerve cord that differentially couple sensory input to motor output.