In this paper, we propose a novel method that treats pose estimation as a problem with the constraints of human segmentation consistency from single images. Different from the previous paper, we integrate pose estimation and object segmentation into a joint optimization. With the support of segmentation consistency, we can obtain more reliable pose results. Through analyzing the energy function of pose estimation and human segmentation, we convert the pose estimation into a binary optimization problem that has the same formation as segmentation. The top-down pose shape cues, bottom-up visual cues, and the consistency constraints that penalize the mismatching of pose and human foreground are incorporated into our final objective function. Qualitative and quantitative experimental results demonstrate the merits of our method in pose estimation on Ramanan benchmark and Buffy data sets.