This paper proposes a dynamic multi-swarm global particle swarm optimization (DMS-GPSO) that consists of two novel strategies to balance the exploration and exploitation abilities. In DMS-GPSO, the entire population is divided into a global sub-swarm and dynamic multiple sub-swarms. During the evolutionary process. the global sub-swarm focus on exploitation under the guidance of the optimal particle in the entire population, while the dynamic multiple sub-swarms focus on exploration under the guidance of the neighbors best-so-far position. Moreover, the store-reset strategy of the global sub-swarm is applied to save computational resource and increase population diversity, aiming to improve the exploration ability of DMS-GPSO at the initial evolutionary stage. At the later evolutionary stage, some favorable particles of the global sub-swarm stored in an archive are combined with particles in the DMS sub-swarms as a single population to search for optimal solutions, intending to enhance the exploitation ability. The comparison results between DMS-GPSO and other 7 peer algorithms on the CEC2013 test suite demonstrate that DMS-GPSO can avoid the premature convergence when solving multimodal problems, and yields more effective performance in complex problems.