We propose the eigenvalue problem of an anisotropic diffusion operator for image segmentation. The diffusion matrix is defined based on the input image. The eigenfunctions and the projection of the input image in some eigenspace capture key features of the input image. An important property of the model is that for many input images, the first few eigenfunctions are close to being piecewise constant, which makes them useful as the basis for a variety of applications such as image segmentation and edge detection. The eigenvalue problem is shown to be related to the algebraic eigenvalue problems resulting from several commonly used discrete spectral clustering models. The relation provides a better understanding and helps developing more efficient numerical implementation and rigorous numerical analysis for discrete spectral segmentation methods. The new continuous model is also different from energy-minimization methods such as geodesic active contour in that no initial guess is required for in the current model. The multi-scale feature is a natural consequence of the anisotropic diffusion operator so there is no need to solve the eigenvalue problem at multiple levels. A numerical implementation based on a finite element method with an anisotropic mesh adaptation strategy is presented. It is shown that the numerical scheme gives much more accurate results on eigenfunctions than uniform meshes. Several interesting features of the model are examined in numerical examples and possible applications are discussed.