Physics-Informed Neural Networks (PINNs) are considered a powerful tool for solving partial differential equations (PDEs), particularly for the groundwater flow (GF) problem. In this paper, we investigate how the deep learning (DL) architecture, within the PINN framework, is connected to the ability to compute a more or less accurate numerical GF solution, so the link ‘PINN architecture - numerical performance’ is explored. Specifically, this paper explores the effect of various DL components, such as different activation functions and neural network structures, on the computational framework. Through numerical results and on the basis of some theoretical foundations of PINNs, this research aims to improve the explicability of PINNs to resolve, in this case, the one-dimensional GF equation. Moreover, our problem involves source terms described by a Dirac delta function, providing insights into the role of DL architecture in solving complex PDEs.