In recent years, multimodal 3D object detection methods have garnered significant attention in autonomous driving systems due to their impressive detection performance. However, most existing research seldom addresses the issues of robustness and performance degradation in dynamic environments due to the difficulty of aligning modal features. In this paper, we introduce an innovative efficient fusion method that integrates time series features to improve the accuracy of 3D object detection through multi-sensor fusion, making it more suitable for dynamic and realistic scenarios such as automated driving, and verifying its robustness. The proposed framework incorporates a Temporal Local Self-Fusion Module (TLSFM) in the LiDAR stream to enrich the representation of LiDAR BEV features. To better align BEV features in image streams and point cloud streams, a Cross-Modal Fusion Alignment (CMFA), is introduced. The Temporal Fusion-CMFA (TF-CMFA) framework which contains TLSFM and CMFA module, demonstrates state-of-the-art performance, achieving a mean average precision (mAP) score of 74.4% and a NuScenes detection score (NDS) of 75.7% on the NuScenes benchmark dataset. Performance improvements recorded on the Waymo dataset, with improvements of +2.1 and +2.3 in the ALL-L1 and ALL-L2 metrics compared to VoxelMamba. Finally, robustness experiments demonstrate the performance of proposed approach under sensor failure conditions, demonstrating its exceptional robustness under such conditions.