Given an off-the-shelf camera, one has the freedom to move the camera or play around with its intrinsic parameters such as zoom or aperture settings. We propose a framework for depth estimation from a set of calibrated images, captured under general camera motion and parameter variation. Our framework considers the practical trade-offs in a camera and hence essentially generalizes the more constrained areas such as lateral or axial stereo, shape from defocus/focus etc. We discuss practical issues where such an approach becomes important. We pose the problem in a MAP framework and compute the depth estimates efficiently using belief propagation (BP). We also incorporate the visibility consideration to handle occlusions. Moreover, we use the vital cue from color image segmentation to constrain the estimation process. Our results demonstrate the effectiveness of our approach to localize discontinuities and handle low-textured regions