Endocardium extraction is an important step in the cardiac diseases analysis. Manual and semi-automatic segmentation suffer from the poor accuracy and time-consuming. A new approach by incorporating the spatial-temporal information was proposed to extract endocardium automatically. Firstly, according to the cardiac chamber movements in systolic and diastolic phases, an adaptive window-based method combining texture features was employed to identify the chamber location. Then, the initial contour was quickly detected. Lastly, a deformable model with the selective ensemble learning method was applied to obtain the final smooth contour. The algorithm was evaluated and compared with other contour extraction methods for real echocardiograms. Experimental results indicated that both the mean absolute difference and the percentage of the area overlap exhibited satisfactory performances. Therefore, this approach is a promising method for echocardiogram segmentation.