Accurate quantification of urban soil organic carbon (SOC) is essential for understanding anthropogenic changes and further guiding effective city managements. Visible and near infrared (vis–NIR) spectroscopy can monitor the SOC content in a time- and cost-effective manner. However, processes and mechanisms dominating the relationships between SOC and spectral data in urban soils remain unknown. The main objective of this paper was to evaluate whether multiple stratification strategies (i.e., based on land-use/land-cover [LULC], pH, and spectral clustering) resulted in better predicted performance for SOC compared to the non-stratified (global) models. Results showed that regarding the non-stratified models, the convolutional neural network (CNN) model exhibited the best performance (validation R2 = 0.73), followed by Cubist (validation R2 = 0.66) and memorybased learning (validation R2 = 0.65). After LULC stratification, Cubist model achieved the best prediction (validation R2 = 0.76), improving the value of ratio of performance to interquartile distance by 0.11 compared to the global CNN model. Areas with high SOC values were mainly located in the city center. Stratification by LULC class is a promising strategy for addressing the impact of the soil-landscape diversity and complexity on vis–NIR spectral estimation of SOC in urban soil spectral library.