Vehicle detection and tracking place a vital role in traffic surveillance system (TSS). In the past, many methods had been introduced and implemented still a challenging issue because of dynamic textures such as rain fall, snow and absence of light. Therefore to overcome these problems introduce a novel tracking algorithm based on background subtraction and morphological operations. Firstly the background method is used to detect the moving objects from the video and then morphological operations are applied to remove the noise regions and obtaining more accurate segmentation results. After vehicle detection, a object-based vehicle tracking method is used for building the correspondence between vehicles detected at different time instants. After vehicle tracking, calculate the vehicle count from video.