The detection and measurement of crack at pixel level is a challenge to existing methods. To overcome this challenge, this paper proposes a convolutional encoder-decoder network (CedNet) to detect crack from image, and the maximum widths and orientations of cracks are measured using image post-processing techniques. To realize this, a database including 1800 crack images (with 761×569 pixel resolution) taken from concrete structures is built. Then the CedNet is designed, trained and validated using the built database. The validating results show 98.90% accuracy, 93.58% precision, 94.73% recall, 93.18% F-measure, 87.23% intersection over union (IoU) of crack and 98.82% IoU of background. Subsequently, the robustness and adaptability of the trained model is tested. To measure true maximum widths and orientations of cracks, a laboratory experiment is carried out to calibrate a relation between ratio (pixel distance / real distance) and field of view (camera's view range on concrete surface included in image) and distance from the smartphone to concrete surface. In the post-processing techniques, the perspective transformation is employed to correct distorted images caused by the existence of the oblique angles between the smartphone and concrete surfaces. Then the maximum widths and orientations of cracks in predicted results are measured respectively using the Euclidean distance transformation and least squares principle. As comparison, two existing deep learning-based crack detection and measurement method are used to examine the performance of the proposed approach. The comparison results show that the proposed method substantiates quite good performance to detect cracks and measure maximum widths and orientations of cracks in our database.