Judgments about another person's visual perspective are impaired when the self-perspective is inconsistent with the other-perspective. This is a robust finding in healthy samples as well as in schizophrenia (SZ). Studies show evidence for the existence of a reverse effect, where an inconsistent other-perspective impairs the self-perspective. Such spontaneous perspective taking processes are not yet explored in SZ. In the current fMRI experiment, 24 healthy and 24 schizophrenic participants performed a visual perspective taking task in the scanner. Either a social or a non-social stimulus was presented and their visual perspectives were consistent or inconsistent with the self-perspective of the participant. We replicated previous findings showing that healthy participants show increased reaction times when the human avatar's perspective is inconsistent to the self-perspective. Patients with SZ, however, did not show this effect, neither in the social nor in the non-social condition. BOLD responses revealed similar patterns in occipital areas and group differences were identified in the middle occipital gyrus. These findings suggest that patients with SZ are less likely to spontaneously compute the visual perspectives of others.