Image diffusion can smooth away noise and small-scale structures while retaining important features, thereby enhancing the performances of many image processing algorithms such as image compression, segmentation and recognition. In this paper, we present a novel diffusion algorithm for which the filtering kernels vary according to the perceptual saliency of boundaries in the input images. The boundary saliency is estimated through a saliency measure which is generally determined by curvature changes, intensity gradient and the interaction of neighboring vectors. The connection between filtering kernels and perceptual saliency makes it possible to remove small-scale structures and preserves significant boundaries adaptively. The effectiveness of the proposed approach is validated by experiments on various medical images including the color Chinese Visible Human data set and gray MRI brain images.