There is little doubt that cognitive processes involved in perceiving and comparing the duration of the sounds played by a musical instrument are very different from those involved, for instance, in recalling when our latest holidays took place. However, despite their clear difference, we ascribe both processes to the time domain, either in the form of a duration or of a mental time travel. Here we maintain that, despite the several meanings of “time” and its intrinsically heterogeneous nature, a common and somehow surprising characteristic of its representation is its spatial nature. Evidence will be provided suggesting that sensory time (the sounds of the instrument), time travel (last holidays) as well as conceptual time are all spatially represented. Whether this spatial format is the same for all “categories” of time is still unclear, as well as it is unclear whether category-specific spatial layouts exist. The possibility that several heterogeneous aspects of time might be all processed in spatial terms allows to better understand a number of apparently disconnected findings within the vast time domain. We will describe the prominent role played by writing habits in giving a direction to this representation. The possibility will be discussed that a more general tendency exists to represent in a spatial format all ordered sequences, as well as abstract concepts.