3D reconstruction with sparse image sets requires more accurate view geometry estimation than a large number of images based 3D reconstruction. In this paper, we have proposed an automatic 3D reconstruction system based on a small set of images which can estimate the view transformation between different views accurately. The proposed system can build a more complete 3D result when only part of the scene has been initially reconstructed (which often appears in sparse image sets).