In recent years, the utilization of artificial intelligence methodologies in computer vision has markedly propelled the advancement of intelligent healthcare. A multimodal medical image segmentation algorithm is proposed by combining patient metadata with a segmentation network, improving its performance and attaining more accuracy in the final diagnostic results. A fusion method utilizing a transformer backbone network is presented to enhance the efficacy of fusion processes for various modalities of medical data. A channel-level cross-fusion module (channel trans) is incorporated during the fusion phase of two modalities to mitigate interference from extraneous elements in the integrated information. The SMESwin UNet backbone network combines vision transformers and convolutional neural networks to produce multi-scale semantic features and attention mechanisms. It simultaneously collects information from global and local perspectives while minimizing model parameters. Exceptional experimental results were obtained on two publicly accessible glandular pathology datasets, with the Dice segmentation performance index reaching 91.41% on Dataset A and 80.6% on Dataset B. This indicates that utilizing a channel transformer to merge the two modalities effectively generalizes, and the combination of convolutional neural networks with vision transformers improves the ability to extract features in medical images.