Pedestrian detection is a crucial problem in human pose recovery and behavior analysis, especially in applications such as visual surveillance, robotics, and drive-assistance systems. Recently, most pedestrian detection approaches of machine learning and signal processing have achieved advanced performance in traditional natural images. However, there exists a limitation on the accuracy in pedestrian detection. The reason behind this is that supporting information for detecting pedestrian is limited. In fact, spectrum besides visible light can provide abundant discriminative information for pedestrian detection. Therefore, it is significative to exploit multi-spectral information for detection task. In this paper, a multi-spectral based pedestrian detection approach is proposed, which not only takes use of the information of red, green and blue (RGB) bands, but also incorporates the information of near-infrared spectrum into the detection process. Latent variable support vector machines (L-SVM) are employed to train the multi-spectral pedestrian detection model. Experiments are implemented on a new dataset containing 1826 multi-spectral image pairs. The experimental results illustrate that utilizing multi-spectral information achieves significant performance improvement in a pedestrian detection task compared with only using RGB information. HighlightsAn efficient pedestrian detection algorithm is presented.A framework of pedestrian detection integrating visible light with near-infrared spectrum is presented.A new multi-spectral pedestrian dataset containing 1826 annotated NIR and RGB image pairs is built.The proposed algorithm has a fast speed and strong robustness and achieves better performance.