Abstract Sediment concentration (SC) monitoring has always been a pressing issue in water resource management, as many existing instruments still face challenges in accurately measuring due to environmental factors and instrument limitations. A robust technology is worth presenting to apply in the field site. This study firstly uses mean-absolute-error (MAE), root-mean-square error (RMSE), correlation coefficient (CC), and Nash–Sutcliffe efficiency coefficient (NSE) to describe the performance of the proposed convolutional neural network (CNN). Moreover, adapting the ensemble learning concept to compare the multiple machine learning (ML) approaches, the CNN presents the highest predicted accuracy, 91%, better than SVM (79%), VGG19 (63%) and ResNet50 (35%). As a result, the proposed CNN framework can appropriately apply the monitoring needs. The primary purpose is to develop a simple, accurate, and stable SC monitoring technology. Instead of some complex architectures, a simple and small neural network is adopted to implement real-time application (RTA). Via our design, such a traditional but critical issue can be improved to a new state. For example, by incorporating the concept of the Internet of Things (IoT) with our design, the distributed computing system for large-scale environmental monitoring can be realized quickly and easily.