Abstract Object tracking algorithms rely on user input to learn the object of interest. In multi-object tracking, this can be a challenge when the user has to provide a lot of locations to track. This paper presents a new approach that reduces the need for user input in multi-tracking. The approach uses density based clustering to analyse the colours in one frame and find the best separation of colours. The colours selected from the detection are learned and used in subsequent frames to track the colours through the video. With this training approach, the user interaction is limited to selecting the colours rather than selecting the multiple location to be tracked. The training algorithm also provides online training even when training on thousands of features.