Grapheme–color synesthesia is defined as the experience of color in relation to letters, words and numbers. This form of synesthesia is a relatively common type and has received the most attention in the research field in terms of the neural basis of the synesthetic experience and behavioral features of synesthesia such as attention and memory. Grapheme–color and linguistic–color synesthesia have a genetic component. Still, language (the most common synesthetic inducer) is acquired through an interaction between genes and the environment. In order to probe this interaction, we compared relatives of grapheme–color synesthetes to a group of matched controls in a reading-in-color training paradigm. All participants read specially prepared books in which four high-frequency letters were paired with four high-frequency colors. Magnetic resonance images (MRI) were acquired before and after training, including diffusion tensor imaging (DTI). In this sample, there was no evidence of increased acquisition of the letter–color associations by the relatives of synesthetes. No group differences were found in behavior, while there are differences in functional activation on the Stroop task. Imaging results showed significant differences in brain activation in regions known to be involved in grapheme–color synesthesia across both groups. These regions include temporal–occipital cortex, inferior and superior parietal lobe, precentral gyrus, and frontal pole. We conclude that the neural basis of the effects of reading in color relies primarily on the same brain regions as the experience of (authentic) grapheme–color synesthesia.