We present a tolerance analysis that is applicable to a large group of stimuli used in structure-from-motion tasks. Human performance in structure-from-motion tasks reflects the fact that the visual system deals with projections of a 3-D world on the retina. A tolerance analysis reveals the relationship between the projections and the 3-D world. Any realistic model of the visual system should incorporate a tolerance analysis as a complete description of the stimulus. By way of example we apply the tolerance analysis to the stimuli used in two widely known experiments in which different properties of structure were tested--that is, perceived nonrigidity (Norman & Todd, 1993) and ordering in depth (Hildreth, Grzywacz, Adelson, & Inada, 1990). The analysis explains qualitatively the results of these experiments, illustrating that the results are to a large extent due to stimulus limitations rather than to mechanistic properties of the visual system. From our analysis it follows that far more sensitive measurements of the optic information are needed to obtain metric structure than affine structure.