In the standard Simon task, participants carry out spatially defined responses to non-spatial stimulus attributes. Responses are typically faster when stimulus location and response location correspond. This effect disappears when a participant responds to only one of the two stimuli and reappears when another person carries out the other response. This social Simon effect (SSE) has been considered as providing an index for action co-representation. Here, we investigated whether joint-action effects in a social Simon task involve mechanisms of action co-representation, as measured by the amount of incorporation of another person's action. We combined an auditory social Simon task with a manipulation of the sense of ownership of another person's hand (rubber hand illusion). If the SSE is established by action co-representation, then the incorporation of the other person's hand into one's own body representation should increase the SSE (synchronous > asynchronous stroking). However, we found the SSE to be smaller in the synchronous as compared to the asynchronous stroking condition (Experiment 1), suggesting that the SSE reflects the separation of spatial action events rather than the integration of the other person's action. This effect is independent of the active involvement (Experiment 2) and the presence of another person (Experiment 3). These findings suggest that the "social" Simon effect is not really social in nature but is established when an interaction partner produces events that serve as a spatial reference for one's own actions.