Chronic venous insufficiency (CVI) is a serious disease characterised by the inability of the veins to effectively return blood from the legs back to the heart. This condition represents a significant public health issue due to its prevalence and impact on quality of life. In this work, we propose a tool to help doctors effectively diagnose CVI. Our research is based on extracting Visual Geometry Group network 16 (VGG-16) features and integrating a new classifier, which exploits mean absolute deviation (MAD) statistics to classify samples. Although simple in its core, it outperforms state-of-the-art method which is known as the CVI-classifier in the literature, and additionally it performs better than the methods such as multi-layer perceptron (MLP), Naive Bayes (NB), and gradient boosting machines (GBM) in the context of VGG-based classification of CVI. We had 0.931 accuracy, 0.888 Kappa score, and 0.916 F1-score on a publicly available CVI dataset which outperforms the state-of-the-art CVI-classifier having 0.909, 0.873, and 0.900 for accuracy, Kappa score, and F1-score, respectively. Additionally, we have shown that our classifier has a generalisation capacity comparable to support vector machines (SVM), by conducting experiments on eight different datasets. In these experiments, it was observed that our classifier took the lead on metrics such as F1-score, Kappa score, and receiver operating characteristic area under the curve (ROC AUC).