AbstractSimultaneous localization and mapping (SLAM) is crucial for autonomous mobile robots. Most of the current SLAM systems are based on an assumption: the environment is static. However, the real environment is full of dynamic elements, such as pedestrians or vehicles, as well as changes in illumination and appearance over time. In this paper, DE‐SLAM, a visual SLAM system that can deal with short‐term and long‐term dynamic elements at the same time is proposed. A novel dynamic detection and tracking module that utilizes both semantic and metric information is proposed, and the localization accuracy is highly improved by eliminating features falling on the dynamic objects. A unified loop detection, loop check and global optimization module is used to perform loop closure. Experimental results on datasets and real environments show that DE‐SLAM outperforms other state‐of‐the‐art SLAM systems in dynamic environments.