With the train’s speed improving, and running interval time decreasing, traditional rail defect inspection methods cannot meet the efficiency requirement of railway industrial standards. In this paper, we propose a new online rail defect inspection method which is using electromagnetic tomography (EMT) technique. The method uses the tomographic approach to measure the alternating magnetic signal modulated by cracks in the rail and then reconstruct the distribution of cracks. The difference to common eddy-current rail defect inspection is, that the EMT can acquire the defect shape and position information. The sensor and signal processing hardware of the EMT rail defect inspection system can be installed on the common passenger or freight train. This solution allows continuous inspection, and therefore reduces the critical accident events caused by rail defect. A model is designed and the forward problem of the model is calculated using electromagnetic finite-element method. Rail defect reconstruction simulations are done using linear backprojection and Tikhonov regularization algorithm to verify the principle of this method. Furthermore, an experimental system is built to simulate the rail defect. The laboratory experiment results show that the electromagnetic rail defect inspection is a feasible approach for reconstruction of the shape and position information of rail defect.