Scholars have classified soil to understand its complex and diverse characteristics. The current trend of precision agricultural technology demands a change in conventional soil identification methods. For example, soil color observed using Munsell color charts is subjective and lacks consistency among observers. Soil classification is essential for soil management and sustainable land utilization, thereby facilitating communication between different groups, such as farmers and pedologists. Misclassified soil can mislead processes; for example, it can hinder fertilizer delivery, affecting crop yield. On the other hand, deep learning approaches have facilitated computer vision technology, where machine-learning algorithms trained for image recognition, comparison, and pattern identification can classify soil better than or equal to human eyes. Moreover, the learning algorithm can contrast the current observation with previously examined data. In this regard, this study implements a convolutional neural network (CNN) model called Soil-MobiNet to classify soils. The Soil-MobiNet model implements the same pointwise and depthwise convolutions of the MobileNet, except the model uses the weight of the pointwise and depthwise separable convolutions plus an additional three dense layers for feature extraction. The model classified the Vellore Institute of Technology Soil (VITSoil) dataset, which is made up of 4864 soil images belonging to nine categories. The VITSoil dataset samples for Soil-MobiNet classification were collected over the Indian states and it is made up of nine major Indian soil types prepared by experts in soil science. With a training and validation accuracy of 98.47% and an average testing accuracy of 93%, Soil-MobiNet showed outstanding performance in categorizing the VITSoil dataset. In particular, the proposed Soil-MobiNet model can be used for real-time soil classification on mobile phones since the proposed system is small and portable.