The semantic segmentation of infrared weak targets relies more on detailed texture features, and deeper network architecture are not suitable for semantic segmentation of infrared targets, making it difficult to accurately segment weak targets from complex backgrounds. This article focuses on the semantic segmentation task requirements of infrared moving targets in complex backgrounds. Based on the publicly available target detection and tracking dataset, an infrared image semantic segmentation dataset is annotated and constructed. The STDC-Seg model is used to optimize the characteristics of infrared images, and an infrared target semantic segmentation algorithm STDC-Infrared is proposed. Redesign the network downsampling structure, adding spatial attention module and multi-scale adaptive fusion module. The experimental results show that compared with STDC-Seg, this algorithm has improved the average intersection to union ratio and average pixel accuracy by 12.47% and 12.55%, respectively, on the infrared image semantic segmentation dataset. In particular, the intersection to union ratio and pixel accuracy of infrared aircraft targets have improved by 31.53% and 35.82%, respectively, effectively improving the performance of semantic segmentation algorithms in complex background infrared moving target scenes.