Directional judgments of the arrow became slower when the direction and location were incongruent in the spatial Stroop task (i.e., a spatial Stroop effect). In contrast, gaze judgments were slower when they were congruent (i.e., a reversed congruency effect). The present study examined the reaction time (RT) distribution of interference effects in the spatial Stroop task to clarify the temporal characteristics of the spatial Stroop effect, which is known to be reversed for social targets, such as gaze direction. Participants responded to the laterally presented target (i.e., arrow, gaze, fish-only, and fish with mosaic) while ignoring its location. The arrow’s spatial Stroop effect decreased as the overall RT increased, reflecting the temporal decay of automatically activated task-irrelevant codes (i.e., location). Critically, the gaze’s reversed congruency effect increased as the overall RT increased. This result supports the dual-stage hypothesis (Tanaka et al., 2023) and reflects the late-arriving selective inhibition of task-irrelevant codes. Similar results were replicated in Experiment 2, which manipulated the complexity of the backgrounds of non-social targets and in the reanalysis of existing data. These findings imply that the interplay between task-irrelevant activation and subsequent inhibition is modulated by specific stimulus characteristics, influencing spatial response selection.