Human physical action classification is an emerging area of research for human-to-machine interaction, which can help to disable people to interact with real world, and robotics application. EMG signals measure the electrical activity muscular systems, which involved in physical action of human. EMG signals provide more information related to physical action. In this paper, we proposed deep transfer learning-based approach of human action classification using surface EMG signals. The surface EMG signals are represented by time–frequency image (TFI) by using short-time Fourier transform. TFI is used as input to pre-trained convolutional neural network models, namely AlexNet and VGG16, for deep feature extraction, and support vector machine (SVM) classifier is used for classification of physical action of EMG signals. Also, the fine-tuning of the pre-trained AlexNet model is also considered. The experimental results show that deep feature extraction and SVM classification method and fine-tuning have obviously improved the classification accuracy when compared with various results from the literature. The 99.04% accuracy score is obtained with AlexNet fc6 + AlexNet fc7 + VGG16 fc6 + VGG16 fc7 deep feature concatenation and SVM classification. 98.65% accuracy score is performed by fine-tuning of the AlexNet model. We also compare the obtained results with some of the existing methods. The comparisons show that the deep feature concatenation and SVM classification method provide better classification accuracy than the compared methods.