Robust Visual Monitoring system that detects and analyze activities based on human, vehicle or animal detection and tracking assures good surveillance. This paper focuses on two dimensional object detection and tracking using Modified Gaussian mixture model (GMM), Adaptive threshold and prediction filtering. Our proposed approach improves learning rate of model for the various constraint like clutter background, light variations, slow and fast moving objects, entering or removed objects, partial occlusion. Our aim is to design such an algorithm which works well on outdoor video sequences.