To develop an integrated full-life cycle monitoring and early warning system for pavement structures. In this research the mentoring system deployment technology for automatic monitoring is carried out on asphalt crack, and asphalt pothole. The data collection and analysis technology based on image recognition technology to achieve automatic collection analysis and early warning of real-time data on the operation status of facilities by deep learning algorithm. The developed an asphalt crack, and asphalt pothole monitoring model is based on a faster region-based convolutional neural network (Faster R-CNN) image classification algorithm. Provided with a relatively heterogeneous dataset, the use of deep learning allows the development of an asphalt crack, and asphalt pothole monitoring system. We confirmed that the new approach really works well through both the numbers and experiments. For the dataset used in this work, modeled with an accuracy, regression, and F-score versus the overall performance are 87.24%, 84.12%, and 85.96%, respectively, highlighting the potential of using deep learning for the monitoring of cracks in pavement surfaces.