During the last years, Automatic video analysis has become a very important research for video management, such as video index and video retrieval. The application domains are disparate, ranging from video surveillance to automatic video annotation for sport videos or TV shots. Whatever the application field, most of the works in video analysis are based on two main approaches: the former based on explicit event recognition, focused on finding high level, semantic interpretations of video sequences, and the latter based on anomaly detection. In this paper, we deals with the first approach, where the final goal is to labeling of recognized video event. In order to achieve the goal of automated analysis and annotate events in videos, we have developed a novel video analysis and trajectory based video annotation system called VATAS. The system involves four main modules, which include global motion estimation, motion object detection, object tracking and video annotation. Experimental results prove the validity of the proposed approach.