Localization of the underwater magnetic sensor arrays plays a pivotal role in the magnetic silencing facility. A localization approach is proposed for underwater sensors based on the optimization of magnetic field gradients in the inverse problem of localization. In the localization system, a solenoid coil carrying direct current serves as the magnetic source. By measuring the magnetic field generated by the magnetic source in different positions, an objective function is established. The position vector of the sensor is determined by a novel multi-swarm particle swarm optimization with dynamic learning strategy. Without the optimization of the magnetic source’s positions, the sensors’ positions, especially in the z-axis direction, struggle to meet the requested localization. A strategy is proposed to optimize the positions of the magnetic source based on magnetic field gradients in the three directions of x, y and z axes. Compared with the former method, the model experiments show that the proposed method could achieve a 10 cm location error for the position type 2 sensor and meet the request of localization.