Current damage detection methods based on model updating and sensitivity Jacobian matrixes show a low convergence ratio and computational efficiency for online calculations. The aim of this paper is to construct a real-time automated damage detection method by developing a theory-assisted adaptive mutiagent twin delayed deep deterministic (TA2-MATD3) policy gradient algorithm. First, the theoretical framework of reinforcement-learning-driven damage detection is established. To address the disadvantages of traditional mutiagent twin delayed deep deterministic (MATD3) method, the theory-assisted mechanism and the adaptive experience playback mechanism are introduced. Moreover, a historical residential house built in 1889 was taken as an example, using its 12-month structural health monitoring data. TA2-MATD3 was compared with existing damage detection methods in terms of the convergence ratio, online computing efficiency, and damage detection accuracy. The results show that the computational efficiency of TA2-MATD3 is approximately 117–160 times that of the traditional methods. The convergence ratio of damage detection on the training set is approximately 97%, and that on the test set is in the range of 86.2%–91.9%. In addition, the main apparent damages found in the field survey were identified by TA2-MATD3. The results indicate that the proposed method can significantly improve the online computing efficiency and damage detection accuracy. This research can provide novel perspectives for the use of reinforcement learning methods to conduct damage detection in online structural health monitoring.