Developing deep learning models often involves working with balanced datasets. However, in real-world scenarios, class equilibrium is rarely observed. To address this, oversampling and undersampling techniques are commonly employed, with appropriate monitoring and countermeasures to mitigate the risk of overfitting. Yet, even when datasets are balanced, they can still introduce biases, leading to better classification performance for certain classes over others. One significant cause of such biases is class underrepresentation, which is challenging to control and often arises from the acquisition of uneven feature distributions influenced by endogenous factors and environmental conditions. In such cases, random data augmentation can inadvertently amplify these discrepancies, exacerbating the issue of underrepresentation. To address this challenge, this paper proposes a metrics-based closed-loop (MbCL) approach that strategically unbalances the dataset to enhance class-wise performance and model generalization in line with the training process. The proposed method iteratively adjusts and improves model performance across different classes by employing data augmentation techniques, including both classical and GAN-based methods, to mitigate classes’ underrepresentation. To compare regular (linear) and MbCL-based approaches, a couple of contextually distinct datasets were considered to broaden consistency analysis across some diversity. Using these datasets, 72 models with varying configurations – including different convolutional neural network architectures, initial learning rates, and optimizers – were initially trained and then evaluated against imagery test sets. In this first assessment, MbCL methods matched or outperformed linear training in approximately 85% of the cases. Additionally, all the models were tested on an external image set acquired under different contextual conditions but with matching labels to the training datasets. The results showed that the proposed MbCL methods consistently outperformed linear training, achieving top-accuracies of 32% and 72% for both the encompassed dataset-related contexts, therefore, also indicating that classic and GAN-based augmentation approaches had a positive impact on inference performances as resampling strategies. Furthermore, complementary indicators, including gradient-weighted class activation mapping (Grad-CAM) and intersection over union (IoU), were analyzed to explore the relationship between performance and visualizable features.