Abstract In the circular restricted three-body problem (CR3BP), transit orbit is a class of orbit which can pass through the bottleneck region of the zero velocity curve and escapes from the vicinity of the primary or the secondary. This kind of orbit plays a very important role in the design of space exploration missions. A kind of low-energy interplanetary transfer, which is called Interplanetary Superhighway (IPS), can be realized by utilizing transit orbits. To use the transit orbit in actual mission design, a key issue is to find an algorithm which can separate the states corresponding to transit orbits from the states corresponding to other types of orbits rapidly. In fact, the distribution of transit orbit in the phase space has been investigated by numerical method, and a Fourier series approximation method has been introduced to describe the boundary of transit orbits. However, the Fourier series approximation method needs several hundred sets of Fourier series. The coefficients of these Fourier series are neither easy to be computed nor convenient to be stored, which makes the method can hardly be used in actual mission design. In this paper, the support vector machine (SVM) is used to classify the trajectories in the CR3BP. Using the Gaussian kernel, the 6-dimensional states in the CR3BP are mapped into an infinite-dimensional space, and the bound of the transit orbits is described by a hyperplane. A training data generation method is introduced, which reduces the size of training data by generating the states near the hyperplane. The numerical results show that the proposed algorithm gives the good correct rate of classification, and its computing speed is much faster than that of the Fourier series approximation method.