The brain converts perceptual information into appropriate patterns of muscle activity depending on the categorization and localization of sensory cues. Sensorimotor information might either be encoded by distributed networks or by "labeled lines" connecting sensory channels to dedicated behavioral pathways. Here we investigate, in the context of natural behavior, how the tectum of larval zebrafish can inform downstream premotor areas. Optogenetic mapping revealed a tectal motor map underlying locomotor maneuvers for escape and approach. Single-cell reconstructions and high-resolution functional imaging showed that two spatially segregated and uncrossed descending axon tracts selectively transmit approach and escape signals to the hindbrain. Moreover, the approach pathway conveys information about retinotopic target coordinates to specific premotor ensembles via spatially ordered axonal projections. This topographic organization supports a tectum-generated space code sufficient to steer orienting movements. We conclude that specific labeled lines guide object-directed behavior in the larval zebrafish brain.