The appearance quality index of tobacco leaves is widely used in the tobacco industry. But in national standards for flue-cured tobacco, the specified indicators only have qualitative descriptions, and a few have a range of quantitative values, lacking quantitative calculation methods, which affects the effective use of these indicators in tobacco automatic grading. In this work, we provided a computer vision-based quantitative research approach for color intensity, length, waste, and body of tobacco appearance quality indicators. We also designed quantitative algorithms for these indices to achieve precise quantitative values to address this issue. Especially we proposed the quantization algorithm of color intensity and waste originally. In order to employ the quantification algorithm for each index, the tobacco leaf image was first segmented to determine the tobacco leaf region in the picture. Second, a mesh segmentation technique for the color intensity is developed. A comparison of the color differences between the several sub-images of the tobacco leaf image is divided. The pixel length was swiftly calculated, the boundary points at both ends of the tobacco leaf were located, the minimum outer rectangle of the tobacco leaf was calculated for the length index, and the actual length was obtained by the checkerboard reference data. Internal waste and marginal waste are the categories under which the waste index is divided. To locate holes and abnormal areas for internal waste, the connected region analysis is employed. The waveform of the edge was created and studied to determine the missing part of the edge. The actual area of the tobacco leaf was calculated by the design algorithm, the body index was expressed as weight per unit area, and the weight was determined by a pressure sensor. Finally, each index’s experimental verification is designed. The empirical findings demonstrate that semantic segmentation average accuracy is 8.4% higher than threshold segmentation in the extraction of tobacco leaf regions. The average relative error between the calculated tobacco leaf length and the manual measurement is 2.83%. The average accuracy of tobacco leaf position classification was 88.52% under the six classifiers. The correlation coefficient between tobacco leaf body quantification value and tobacco leaf thickness value is 0.9270.