In animals with binocular vision, retinal fibers either project across the midline or they remain on the same side of the ventral diencephalon, forming an X-shaped commissure known as the optic chiasm. The correct formation of the optic chiasm during development is essential to establish a fully functional visual system. Visual dysfunction associated with axonal misrouting at the optic chiasm has been described in albino individuals and in patients with non-decussating retinal-fugal fiber syndrome. Although little is known about the causes of retinal misrouting in these conditions, the molecular mechanisms responsible for the formation of the optic chiasm are beginning to be elucidated in vertebrates. This review focuses on our current knowledge of how the optic chiasm forms, which will hopefully help us to better understand these congenital anomalies.