In this paper, we attempt to make a distinction between egocentrism and allocentrism in social cognition, based on the distinction that is made in visuo-spatial perception. We propose that it makes a difference to mentalizing whether the other person can be understood using an egocentric ("you") or an allocentric ("he/she/they") stance. Within an egocentric stance, the other person is represented in relation to the self. By contrast, within an allocentric stance, the existence or mental state of the other person needs to be represented as independent from the self. We suggest here that people with Asperger syndrome suffer from a disconnection between a strong naïve egocentric stance and a highly abstract allocentric stance. We argue that the currently used distinction between first-person and third-person perspective-taking is orthogonal to the distinction between an egocentric and an allocentric stance and therefore cannot serve as a critical test of allocentrism.