Properly joining the different modes of a multiregime tracker is difficult. This paper describes a novel, image-based algorithm for tracking a maneuvering target. The form of the algorithm is like the extended Kalman filter, but the gain adjustment utilizes image information in a sophisticated manner to better compute the error covariance. The performance of this filter is compared with two alternatives: an extended Kalman filter without image augmentation and an extended Kalman filter with acceleration aiding and simple, image-derived gain scheduling. The response of the proposed tracker is shown by example to be superior to the others. Although too complex to permit online implementation, the performance of the algorithm provides a useful bound on tracker fidelity attainable with an image-augmented architecture.