Face perception is mediated by a distributed neural system in the human brain. Attention, memory and emotion modulate the neural activation evoked by faces, however the effects of gender and sexual orientation are currently unknown. To test whether subjects would respond more to their sexually-preferred faces, we scanned 40 hetero- and homosexual men and women whilst they assessed facial attractiveness. Behaviorally, regardless of their gender and sexual orientation, all subjects similarly rated the attractiveness of both male and female faces. Consistent with our hypothesis, a three-way interaction between stimulus gender, beauty and the sexual preference of the subject was found in the medial orbitofrontal cortex (OFC). In heterosexual women and homosexual men, attractive male faces elicited stronger activation than attractive female faces, whereas in heterosexual men and homosexual women, attractive female faces evoked stronger activation than attractive male faces. These findings suggest that the OFC represents the value of salient sexually-relevant faces, irrespective of their reproductive fitness.