Contour detection is an important and fundamental problem in computer vision which finds numerous applications. Despite significant progress has been made in the past decades, contour detection from natural images remains a challenging task due to the difficulty of clearly distinguishing between edges of objects and surrounding backgrounds. To address this problem, we first capture multi-scale features from pixel-level to segmentlevel using local and global information. These features are mapped to a space where discriminative information is captured by computing posterior divergence of Gaussian mixture models and then used to train a random forest classifier for contour detection. We evaluate the proposed algorithm against leading methods in the literature on the Berkeley segmentation and Weizmann horse data sets. Experimental results demonstrate that the proposed contour detection algorithm performs favorably against state-of-the-art methods in terms of speed and accuracy.