Non-destructive thermography is a high-speed, low-cost, and safe solution for subsurface defects detection of carbon fiber reinforced polymer (CFRP) materials, providing essential quality control in aerospace, automobile, and sports industries. In this paper, we build a reflective lock-in thermography system and construct a dataset that contains real-captured thermal image sequences of CFRP samples with various simulated internal defects under different excitation frequencies. Then, we present a novel 3D Convolutional Neural Network (CNN) model incorporating a combination of spatial and temporal convolutional filters and batch-size independent Group Normalization (GN) as a unified framework to process thermal image sequences captured by lock-in thermography for simultaneous subsurface defect detection and depth estimation. Finally, we define a multi-task loss function to perform end-to-end training of both defect detection and depth estimation tasks based on the real-captured infrared sequences. Comparative experiments are carried out on CFRP specimens with artificial defects of various sizes/shapes and at different depths. Qualitative and quantitative results illustrate that our 3D CNN model is capable of predicting accurate locations and depths of subsurface defects and performs favorably against the hand-crafted and CNN-based methods in lock-in thermography for individual defect detection and depth estimation tasks. The captured dataset and the source codes will be made publicly available.