This paper proposes a novel method for sequence modelling which we call Seq2Space. The basic idea is to project sequential information into the channel dimension. The Seq2Space layer outperforms the Transformer on every dataset contained in the Long Range Arena (LRA) benchmark as well as on the WSJ0-2 Mix benchmark for single-channel speech separation. Compared to previous methods which were tested on the LRA, the proposed Seq2Space layer does not quite reach the accuracy of the convolution-based methods. It is, however, more than twice as fast as the next fastest method as well as the most memory efficient, and still reaches an average accuracy of 71.15%. On the WSJ0-2Mix, the Seq2Space layer outperforms all other sequence modelling methods in our experiments except for the MEGA layer. By replacing Transformers with the Seq2Space layer on a current SOTA method, we are able to reach 22.8 dB SI-SDR improvement, which is comparable to current SOTA while being significantly faster and more memory efficient during both training and inference.