Deep neural network (DNN) has been used as a learning model for modeling the hierarchical architecture of human brain. However, DNN suffers from problems of learning efficiency and computational complexity. To address these problems, deep sparse autoencoder network (DSAN) is used for learning facial features, which considers the sparsity of hidden units for learning high-level structures. Meanwhile, Softmax regression (SR) is used to classify expression feature. In this paper, Softmax regression-based deep sparse autoencoder network (SRDSAN) is proposed to recognize facial emotion in human-robot interaction. It aims to handle large data in the output of deep learning by using SR, moreover, to overcome local extrema and gradient diffusion problems in the training process, the overall network weights are fine-tuned to reach the global optimum, which makes the entire depth of the neural network more robust, thereby enhancing the performance of facial emotion recognition. Results show that the average recognition accuracy of SRDSAN is higher than that of the SR and the convolutional neural network. The preliminarily application experiments are performed in the developing emotional social robot system (ESRS) with two mobile robots, where emotional social robot is able to recognize emotions such as happiness and angry.