The problem of deciding whether two pixels in an image have the same real world color is a fundamental problem in computer vision. Many color spaces are used in different applications for discriminating color from intensity to create an informative representation of color. The major drawback of all of these representations is that they assume no color distortion. In practice the colors of real world images are distorted both in the scene itself and in the image capturing process. In this work we introduce color lines, an image specific color representation that is robust to color distortion and provides a compact and useful representation of the colors in a scene.