Lossless recovery in visual secret share (VSS) is very meaningful. In this paper, a novel lossless recovery algorithm for the basic matrix VSS is proposed. The secret image is reconstructed losslessly by using simple exclusive XOR operation and merging pixel. The algorithm not only can apply to the VSS without pixel expansion but also can apply to VSS with pixel expansion. The condition of lossless recovery of a VSS is given by analyzing the XOR all columns of basic matrixes. Simulations are conducted to evaluate the efficiency of the proposed scheme.