AbstractSynesthetes can be distinguished from nonsynesthetes on a variety of experimental tasks because their concurrent synesthetic experiences can affect task performance if these experiences match or conflict with some aspect of the stimulus. Here, we tested grapheme‐color synesthetes and nonsynesthetic control participants using a novel perceptual similarity task to assess whether synesthetes’ concurrent color experiences influence perceived grapheme similarity. Participants iteratively arranged graphemes and, separately, their associated synesthetic colors in a display, such that similar items were placed close together and dissimilar items further apart. The resulting relative inter‐item distances were used to calculate the pair‐wise (dis)similarity between items in the set, and thence to create separate perceptual representational dissimilarity matrices (RDMs) for graphemes and colors, on an individual basis. On the assumption that synesthetes’ similarity judgments for graphemes would be influenced by their concurrent color experiences, we predicted that grapheme and color RDMs would be more strongly correlated for synesthetes than nonsynesthetes. We found that the mean grapheme‐color RDM correlation was indeed significantly higher in synesthetes than nonsynesthetes; in addition, synesthetes’ grapheme‐color RDM correlations were more likely to be individually statistically significant, even after correction for multiple tests, than those of nonsynesthetes. Importantly, synesthetes’ grapheme‐color RDM correlations scaled with the consistency of their grapheme‐color associations as measured by their Synesthesia Battery (SB) scores. By contrast, the relationship between SB scores and grapheme‐color RDM correlations for nonsynesthetes was not significant. Thus, dissimilarity analysis quantitatively distinguished synesthetes from nonsynesthetes, in a way that meaningfully reflects a key aspect of synesthetic experience.