Electrophysiological source imaging (ESI) is a challenging technique for noninvasively measuring brain activity, which involves solving a highly ill-posed inverse problem. Traditional methods attempt to address this challenge by imposing various priors, but considering the complexity and dynamic nature of the brain activity, these priors may not accurately reflect the true attributes of brain sources. In this study, we propose a novel deep learning-based framework, spatiotemporal source imaging network (SSINet), designed to provide accurate spatiotemporal estimates of brain activity using electroencephalography (EEG).SSINet integrates a residual network (ResBlock) for spatial feature extraction and a bidirectional LSTM for capturing temporal dynamics, fused through a Transformer module to capture global dependencies. A channel attention mechanism is employed to prioritize active brain regions, improving both the accuracy of the model and its interpretability. Additionally, a weighted loss function is introduced to address the spatial sparsity of the brain activity.We evaluated the performance of SSINet through numerical simulations and found that it outperformed several state-of-the-art ESI methods across various conditions, such as varying numbers of sources, source range, and signal-to-noise ratio levels. Furthermore, SSINet demonstrated robust performance even with electrode position offsets and changes in conductivity. We also validated the model on three real EEG datasets: visual, auditory, and somatosensory stimuli. The results show that the source activity reconstructed by SSINet aligns closely with the established physiological basis of brain function.SSINet provides accurate and stable source imaging results.