Stereopsis is a process by which the visual system gauges the relative depth of objects in three-dimensional space by measuring minute positional differences between left and right images. According to the standard notion, this information is thought to be encoded in the primary visual cortex by differences in receptive field (RF) positions for the two eyes. We have developed in alternative model by which stereoscopic information is coded and transformed through a hierarchical chain of processing in the primary visual cortex. Initially, first-order neurons of the visual cortex, simple cells, encode depth information by a scheme based on differences in internal receptive field structure between left and right eyes. Further abstraction of information is achieved by a subset of second-order neurons, complex cells, that are well suited for the detection of depth information in a manner unaffected by positional variations of objects. We review physiological evidence from studies of the cat and monkey that are relevant to the proposed scheme.