In this paper, we consider a mission-critical remote state estimation system with asynchronous massive access of the IoT sensors. We focus on remote state estimation stability of the system in the presence of asynchronous access of the sensors. Exploiting the sparsity in the observation matrix induced by the asynchronous access, we propose a low complexity 2-D message passing state estimation algorithm, where the cyclic loops in the 2-D factor graphs are removed based on the Gaussian-elimination-based quasi-diagonalization of the oversampled aggregated channel matrix of the IoT sensors. As a result, the proposed state estimation scheme is of low complexity and can achieve exact MAP estimation. Using Lyapunov drift analysis, we derive closed-form necessary and sufficient conditions for stability of the mission-critical remote state estimation system. We show that our proposed scheme can achieve significant performance gain over various state-of-the-art baselines for the large-scale system under asynchronous massive access.