This paper presents an approach to determine the optimal stitching line and propagate deformation in irregular overlap for image stitching. First, determine the optimal stitching line that has the lowest cost of some candidate lines in the irregular region; second, detect and match the features in the neighborhood of the optimal stitching line and calculate deformation vectors; finally, spread the deformation vectors to the whole of image region by Poisson equation and constitute the image gradient field that can reconstruct seamless mosaic. This method has no limitation to the shape of the overlapping area so can reduce the juncture of structure and color in the stitching images. Compared with other methods this method can achieve better visual results both in structure and color.