Due to its simplicity and effectiveness in solving many optimization problems, Particle Swarm Optimization (PSO) has attracted the attention of many researchers in the last few years. Nonetheless, in more complicated problems (involving multi-modality, non-separable, etc.), the use of PSO becomes limited and sometimes impractical. In this paper, we proposed an algorithm which is able to deal with optimization problems having several features. More specific, we introduce a refine mechanism into the evolutionary process of PSO for deep exploration of the local search space in which a particle is located. The proposed mechanism is inspired by the animal foraging behaviour, where searching is a mixture of systematic and random movements. In contrast to other existing PSO variants which aimed to improve the exploration ability by using random walk, the proposed approach exploits the locality of the particles by performing local variations in the flight of the individuals according to a Gaussian distribution. In our study, we analyze the effects of the proposed refinement mechanism when it is coupled into different PSO variants which are adopted in our experimental analysis. We show that our proposed approach not only was able to outperform the adopted PSO variants, but also was significantly better in most of the test functions employed in our comparative study.