In the Industry 4.0 era, the visualization and real-time automatic monitoring of smart cities supported by the Internet of Things is becoming increasingly important. The use of filtering algorithms in smart city monitoring is a feasible method for this purpose. However, maintaining fast and accurate monitoring in complex surveillance environments with restricted resources still remains a major challenge. Since the cognitive theory in visual monitoring is difficult to realize in practice, efficient monitoring of complex environments is accordingly hard to be achieved. Moreover, current monitoring methods do not consider the particularities of the human cognitive system, so the re-monitoring ability of the process/target is weak in case of monitoring failure by the monitoring system. In order to overcome these issues, this paper proposes a novel human Short-Long Cognitive Memory mechanism for video surveillance in smart cities. In this mechanism, a memory with a high reliability target is used as a “Long-Term Memory”, whereas a memory with a low reliability target is used as a “Short-Term Memory”. During the monitoring process, the “Short-Term Memory” and “Long-Term Memory” alternation strategy is combined with the stored target appearance characteristics, ensuring that the original model in the memory will not be contaminated or mislaid by changes in the external environment (occlusion, fast motion, motion blur, and background clutter). Extensive simulations showcase that the algorithm proposed in this paper not only improves the monitoring speed without hindering its real-time operation, but also monitors and traces the monitored target accurately, ultimately improving the robustness of the detection in complex scenery, and enabling its application to IoT-assisted smart cities.