Automatic modulation recognition (AMR) of radiation source signals is a research focus in the field of cognitive radio. However, the AMR of radiation source signals at low SNRs still faces a great challenge. Therefore, the AMR method of radiation source signals based on two-dimensional data matrix and improved residual neural network is proposed in this paper. First, the time series of the radiation source signals are reconstructed into two-dimensional data matrix, which greatly simplifies the signal preprocessing process. Second, the depthwise convolution and large-size convolutional kernels based residual neural network (DLRNet) is proposed to improve the feature extraction capability of the AMR model. Finally, the model performs feature extraction and classification on the two-dimensional data matrix to obtain the recognition vector that represents the signal modulation type. Theoretical analysis and simulation results show that the AMR method based on two-dimensional data matrix and improved residual network can significantly improve the accuracy of the AMR method. The recognition accuracy of the proposed method maintains a high level greater than 90% even at −14 dB SNR.