This paper reports on tracking of multiple objects using color histogram backprojection and motion cues. Four tasks which facilitate this are discussed. The first is an adaptive color histogram backprojection (which builds upon the works of Swain and Ballard) and its application to tracking of multiple objects in video sequences. The second task is designing efficient fast blob detectors for selecting regions of interest in video sequences. The third is motion detection based on color histogram backprojection. Achieving these tasks led to multi-objects tracking. Various video sequences were used to demonstrate effective tracking of multiple objects. Notably, we created an interactive multiple objects tracker (CLICK-IT) which in its present form is set at three objects but can be extended easily. CLICK-IT (CSIRO Laboratory for Imaging by Content and Knowledge?Interactive Television) is a PC-based system which provides the user with an intelligent highlighter pen for sports action replay. It is intended as a truly interactive improvement on the drawing pad technology currently used for video annotation in sports broadcasting. The system uses computer vision techniques to focus attention and track particular objects (player(s), ball, horse(s), ?) and semi-automatically annotate the dynamic scene. This paper describes the system including the user interface, the tracking technology based on color and motion information, and system performance evaluation in applications to surveillance-like sequences, running, rugby league football, basketball and soccer. Finally, video scene detection based on color histogram is discussed.