Nowadays, neural networks are considered one of the most effective methods for various tasks such as anomaly detection, computer-aided disease detection, or natural language processing. However, these networks suffer from the "black-box" problem which makes it difficult to understand how they make decisions. In order to solve this issue, an R package called neuralGAM is introduced. This package implements a neural network topology based on Generalized Additive Models, allowing to fit an independent neural network to estimate the contribution of each feature to the output variable, yielding a highly accurate and interpretable deep learning model. The neuralGAM package provides a flexible framework for training Generalized Additive Neural Networks, which does not impose any restrictions on the neural network architecture. We illustrate the use of the neuralGAM package in both synthetic and real data examples.