The complex transcriptional regulatory relationships among genes influence gene expression levels and play a crucial role in determining cellular phenotypes. In this study, we propose a novel, distributed, large-scale transcriptional regulatory neural network model (DLTRNM), which integrates prior knowledge into the reconstruction of pre-trained machine learning models, followed by fine-tuning. Using Saccharomyces cerevisiae as a case study, the curated transcriptional regulatory relationships are used to define the interactions between transcription factors (TFs) and their target genes (TGs). Subsequently, DLTRNM is pre-trained on pan-transcriptomic data and fine-tuned with time-series data, enabling it to accurately predict regulatory correlations. Additionally, DLTRNM can help identify potential key TFs, thereby simplifying the complex and interrelated transcriptional regulatory networks (TRNs). It can also complement previously reported transcriptional regulatory subnetworks. DLTRNM provides a powerful tool for studying transcriptional regulation with reduced computational demands and enhanced interpretability. Thus, this study marks a significant advancement in systems biology for understanding the complex transcriptional regulation within cells.