In this paper we introduce a novel low dimensional method to perform topic detection and classification in Twitter. The proposed method first employs Joint Complexity to perform topic detection. Then, based on the nature of the data, we apply the theory of Compressive Sensing to perform topic classification by recovering an indicator vector, while reducing significantly the amount of information from tweets. In this paper we exploit datasets in various languages collected by using the Twitter streaming API, and achieve increased classification accuracy when comparing to state-of-the-art methods based on bag-of-words, along with several reconstruction techniques.