High-resolution line scan cameras with wide-angle lenses are highly accurate and efficient for tunnel detection. However, due to the curvature of the tunnel, there are variations in object distance that exceed the depth of field of the lens, resulting in uneven defocus blur in the captured images. This can significantly affect the accuracy of defect recognition. While existing deblurring algorithms can improve image quality, they often prioritize results over inference time, which is not ideal for high-speed tunnel image acquisition. To address this issue, we developed a lightweight tunnel structure defect deblurring network (TSDDNet) for curved-tunnel line scanning with wide-angle lenses. Our method employs an innovative progressive structure that balances network depth and feature breadth to simultaneously achieve good performance and short inference time. The proposed depthwise ResBlocks significantly improves the parameter efficiency of the network. Additionally, the proposed feature refinement block captures the structurally similar features to enhance the image details, increasing the peak signal-to-noise ratio (PSNR). A raw dataset containing tunnel blur images was created using a high-resolution line scan camera and used to train and test our model. TSDDNet achieved a PSNR of 26.82 dB and a structural similarity index measure of 0.888, while using one-third of the parameters of comparable alternatives. Moreover, our method exhibited a higher computational speed than that of conventional methods, with inference times of 8.82 ms for a single 512 × 512 pixel image patch and 227.22 ms for completely processing a 2048 × 2560 pixel image. The test results indicated that the structural scalability of the network allows it to accommodate large inputs, making it effective for high-resolution images.