The major objective of image retargeting algorithms is to preserve the viewer's perception while adjusting the aspect ratio of an image. This means that an ideal retargeting algorithm has to be able to preserve high-level semantics and avoid generating low-level image distortion. Stereoscopic image retargeting poses a even more challenging problem in that the 3D perception has to be preserved as well. In this paper, we propose an algorithm based on high-order two-view co-labeling to simultaneously retarget a given stereo pair and preserve its 2D as well as 3D quality. Our experimental results qualitatively demonstrate the improved ability of preserving 2D image structures in both views. In addition, we show quantitatively that our algorithm improves upon the state-of-the-art up to 85% in terms of a measurement based on depth distortion.