In a new integration, we show that the visual-spatial structuring of time converges with auditory-spatial left-right judgments for time-related words. In Experiment 1, participants placed past and future-related words respectively to the left and right of the midpoint on a horizontal line, reproducing earlier findings. In Experiment 2, neutral and time-related words were presented over headphones. Participants were asked to indicate whether words were louder on the left or right channel. On critical experimental trials, words were presented equally loud binaurally. As predicted, participants judged future words to be louder on the right channel more often than past-related words. Furthermore, there was a significant cross-modal overlap between the visual-spatial ordering (Experiment 1) and the auditory judgments (Experiment 2), which were continuously related. These findings provide support for the assumption that space and time have certain invariant properties that share a common structure across modalities.