This paper presents a novel image-based approach for updating the geometry of 3D models. The technique can cope with large-scale models, using a single imaging sensor to which an arbitrary motion is applied. Current approaches usually do not fully take advantage of strong prior information, often available in the form of an initial model. The approach is thus novel in that geometric anomalies are quickly detected, significantly reducing problem complexity. Hence, given a geometric model and known camera motion, the image warping can be calculated and intensity patterns can be predicted. If predictions do not match observations, the model is assumed to be incorrect. The updating is then cast as an optimization problem where differences between observations and predictions are minimized. The algorithm is tested against both synthetic and real imaging data to update a terrain model. Results show that the algorithm can automatically detect and correct geometrical problems of different types and sizes.