In order to reduce the mismatch rate of binocular stereo matching algorithm, a stereo matching algorithm based on improved Census transform is proposed in this paper. Firstly, the classical Census transform is improved, the mean value rather than the center pixel is used as the reference value, and the three states based on the pixels' distances are used to describe the relative relationship between each pixel in the transform window and the reference value; Secondly, the sparse transform window is used to reduce the computational complexity of the algorithm, and the transformed Hamming distance is calculated as the matching cost; Finally, in the maximum disparity range, the points with the least matching cost are selected as the matching points by WTA algorithm, and the disparity is refined by sub-pixel fitting. Experimental results show that the improved Census transformed image has more detailed information in the depth discontinuous region. Compared with the algorithm that directly uses the gray feature as the matching base elements, the proposed algorithm has better operation speed and matching accuracy.