The objective of knowledge graph completion is to comprehend the structure and inherent relationships of domain knowledge, thereby providing a valuable foundation for knowledge reasoning and analysis. However, existing methods for knowledge graph completion face challenges. For instance, rule-based completion methods exhibit high accuracy and interpretability, but encounter difficulties when handling large knowledge graphs. In contrast, embedding-based completion methods demonstrate strong scalability and efficiency, but also have limited utilisation of domain knowledge. In response to the aforementioned issues, we propose a method of pre-training and inference for knowledge graph completion based on integrated rules. The approach combines rule mining and reasoning to generate precise candidate facts. Subsequently, a pre-trained language model is fine-tuned and probabilistic structural loss is incorporated to embed the knowledge graph. This enables the language model to capture more deep semantic information while the loss function reconstructs the structure of the knowledge graph. This enables the language model to capture more deep semantic information while the loss function reconstructs the structure of the knowledge graph. Extensive tests using various publicly accessible datasets have indicated that the suggested model performs better than current techniques in tackling knowledge graph completion problems.