This study aims to investigate whether the emotional response to a picture varies with regard to the change of its color modes. Furthermore, it was examined whether the age difference, in particular, adults and children has an effect. In order to examine the influence of color mode on the emotional responses, picture stimuli were collected when their semantic contents were strongly attached to typical colors, i.e. memory colors. The International Affective Picture System (IAPS) was referred to collect emotionally normative pictures, and eight pictures were selected: two of each quadrant in emotion space defined by pleasure and arousal. The color modes of the pictures were processed into in monotone scales using memory color, gray, and complementary color of its memory color. In experiments, a set of pictures was shown to the subjects who were asked to assess their emotional responses in terms of Pleasure and Arousal using the Self-Assessment-Manikins (SAM). Two identical experiments were carried out with a group of university students and a group of elementary school children. Based on the SAM ratings of both subjects groups, it was revealed that the complementary color can elicit negative emotional responses to the picture that were previously categorized to induce the positive emotion. In addition, when the pictures were presented in grayscale, the emotional response in arousal dimension appeared to be lower. The emotional responses from children appear in a similar pattern as those from adults on Pleasure dimension (p>0.05, Twoway ANOVA). Regarding the emotional responses on Arousal, age difference effects, however no systematic explanation is yet provided.