Recent years have witnessed the significant success of representation learning and deep learning in various prediction and recognition applications. Most of these previous studies adopt the two-phase procedures, namely the first step of representation learning and then the second step of supervised learning. In this process, to fit the training data the initial model weights, which inherits the good properties from the representation learning in the first step, will be changed in the second step. In other words, the second step leans better classification models at the cost of the possible deterioration of the effectiveness of representation learning. Motivated by this observation we propose a joint framework of representation and supervised learning. It aims to learn a model, which not only guarantees the "semantics" of the original data from representation learning but also fit the training data well via supervised learning. Along this line we develop the model of semi-supervised Auto encoder under the spirit of the joint learning framework. The experiments on various data sets for classification show the significant effectiveness of the proposed model.