The proposed scheme is an approach which can be used to improve the performance of traditional image segmentation systems. The scheme is based on a framework that employs the output of an existing image segmentation process together with hierarchical clustering using an information theoretic similarity measure. Experimental results clearly show that when the scheme operates in conjunction with a state of the art image segmentation algorithm, it yields significantly superior performance over a wide spectrum of natural images. These results are based on informal subjective evaluation tests as well as on objective measurements obtained from processing the Berkeley BSDS 300 image dataset.