Road infrastructure maintenance is a critical challenge faced by transportation authorities worldwide. Timely detection and classification of road defects such as potholes, cracks, and surface deterioration are essential for ensuring vehicle safety and reducing maintenance costs. Despite advancements in computer vision, existing automated systems struggle with handling diverse defect types under varying lighting and environmental conditions. This study proposes an Explainable AI (XAI)-based road defect classification system using image processing techniques to provide transparent and accurate identification of pavement anomalies. We compiled a comprehensive road surface image dataset from publicly available sources including RDD2022 and CrackForest. Features were extracted and optimized using pre-trained convolutional neural network (CNN) architectures including ResNet-50 and EfficientNet-B4, addressing high dimensionality and computational complexity issues commonly encountered in pavement image analysis. Explainability techniques such as Grad-CAM and LIME were integrated to ensure transparent, human-interpretable classification outputs.