This paper proposes a hybrid approach that combines intelligent algorithms and modular design to solve a foraging problem within the context of swarm robotics. Deep reinforcement learning (RL) and particle swarm optimization (PSO) are deployed in the proposed modular architecture. They are utilized to search for many resources that vary in size and exhibit a dynamic nature with unpredictable movements. Additionally, they transport the collected resources to the nest. The swarm comprises 8 E-Puck mobile robots, each equipped with light sensors. The proposed system is built on a 3D environment using the Webots simulator. Through a modular approach, we address complex foraging challenges characterized by a non-static environment and objectives. This architecture enhances manageability, reduces computational demands, and facilitates debugging processes. Our simulations reveal that the RL-based model outperforms PSO in terms of task completion time, efficiency in collecting resources, and adaptability to dynamic environments, including moving targets. Notably, robots equipped with RL demonstrate enhanced individual learning and decision-making abilities, enabling a level of autonomy that fosters collective swarm intelligence. In PSO, the individual behavior of the robots is more heavily influenced by the collective knowledge of the swarm. The findings highlight the effectiveness of a modular design and deep RL for advancing autonomous robotic systems in complex and unpredictable environments.