This paper presents a segmentation method that can automatically segment a scene into its constitute objects. The proposed method is consists of four major modules: spatial segmentation, temporal segmentation, object extraction and tracking. For the spatial segmentation, a video sequence is modeled using Markov random fields (MRFs), and the energy function of each MRF is minimized by chromosomes that evolve using distributed genetic algorithms (DGAs). Then, to improve the performance, chromosomes of the subsequent frame are started with the segmentation result of the previous frame, thereafter only unstable chromosomes corresponding to the actually moving objects parts are evolved by mating. The change detection masks are produces by the temporal segmentation, and video objects are extracted by combining two segmentation results. Finally, the extracted objects are tracked using the proposed tracking algorithm. Here, the proposed object tracking method need not to compute the motion field or motion parameters. It can deal with scenes including multiple objects, plus keep track of objects even when they stop moving for an arbitrarily long time. The results tested with several real video sequences show the effectiveness of the proposed method.