Based on the classic least squares twin support vector machine (LSTSVM), an efficient but simple Least Squares Twin Support Vector Machine-Partial Binary Tree (LSTSVM-PBT)for binary classification problem was proposed. This algorithm introduces binary tree into LSTSVM, the problem summed up as binary tree classification for each data ultimately. Compared to traditional SVM, LSTSVM-PBT has low time complexity. Reliable theoretical analysis and extensive experiments show that LSTBSVM-PBT is fast computationally and obtain the higher performance than traditional algorithm.