Protanopes and deuteranopes, despite lacking a chromatic dimension at the receptor level, use the color terms "red" and "green", together with "blue" and "yellow", to describe their color percepts. Color vision models proposed so far fail to account for these findings in dichromats. We confirmed, by the method of hue scaling, the consistent use of these color terms, as well as their dependence on intensity, in subjects shown to have only a single X-chromosomal opsin gene each. We present a model for the processing of photoreceptor signals which, under physiologically plausible assumptions, achieves a trichromat-like representation of dichromatic receptor signals. Key feature of the dichromat model is the processing of the photoreceptor signals in parallel channels with different gains and nonlinearities. In this way, the two-dimensional receptor signals are represented on a manifold in a higher-dimensional space, supporting categorization for efficient image segmentation. Introducing a third cone opsin yields a model that explains normal, trichromat hue scaling.