The authors describe an algorithm for implementing a multisensor system in a model-based environment with consideration of the constraints. Based on an environment model, geometric features and constraints are generated from a CAD model database. Sensor models are used to predict sensor response to certain features and to interpret raw sensor data. A constrained MMS (minimum mean squared) estimator is used to recursively predict, match, and update feature location. The effects of applying various constraints in estimation were shown by simulation system mounted on a robot arm for localization of known object features. >