Background Driving is a complex skill involving various cognitive activities. Previous research has explored differences in the brain structures related to the navigational abilities of drivers compared to non-drivers. However, it remains unclear whether changes occur in the structures associated with low-level sensory and higher-order cognitive abilities in drivers. Methods Gray matter volume, assessed via voxel-based morphometry analysis of T1-weighted images, is considered a reliable indicator of structural changes in the brain. This study employs voxel-based morphological analysis to investigate structural differences between drivers (n = 22) and non-drivers (n = 20). Results The results indicate that, in comparison to non-drivers, drivers exhibit significantly reduced gray matter volume in the middle occipital gyrus, middle temporal gyrus, supramarginal gyrus, and cerebellum, suggesting a relationship with driving-related experience. Furthermore, the volume of the middle occipital gyrus, and middle temporal gyrus, is found to be marginally negative related to the years of driving experience, suggesting a potential impact of driving experience on gray matter volume. However, no significant correlations were observed between driving experiences and frontal gray matter volume. Conclusion These findings suggest that driving skills and experience have a pronounced impact on the cortical areas responsible for low-level sensory and motor processing. Meanwhile, the influence on cortical areas associated with higher-order cognitive function appears to be minimal.