AbstractWe investigate the emotional response to colors in ordinary multicolored images. In psychophysical experiments, using both category scaling and interval scaling, observers are asked to judge images using three emotion factors: activity, weight, and heat. The color emotion metric was originally developed for single colors, and later extended to include pairs of colors. The same metric was recently used in image retrieval. The results show that people in general perceive color emotions for multi‐colored images in similar ways, and that observer judgments correlate with the recently proposed method used in image retrieval. The intended usage is in retrieval systems publicly available on the Internet, where both the user and the viewing environment is unknown, which requires novel ways of conducting the psychophysical experiments. © 2010 Wiley Periodicals, Inc. Col Res Appl, 36, 210–221, 2011;