This paper faces the automatic object tracking problem in a video-surveillance task. A previously selected and then identified target has to be retrieved in the scene under investigation because it is lost due to masking, occlusions, or quick and unexpected movements. A two-step procedure is used, firstly motion detection is used to determine a candidate target in the scene, secondly using a semantic categorization and Content Based Image Retrieval techniques, the candidate target is identified whether it is the one that was lost or not. The use of Content Based Image Retrieval serves as support to the search problem and is performed using a reference data base which was populated a priori.