Due to the impressive high‐speed, correlation filters began to receive more attention. In this Letter, a real time target tracking framework based on the kernelised correlation filter is proposed. The framework includes two parts: a boosted fusion feature colour names‐histograms of oriented gradients (CN‐HOG) that consists of CNs and HOG, and a novel fragments‐based tracking algorithm that is robust to partial occlusion. Experiments with the evaluation of state‐of‐the‐art single object trackers show the effectiveness and speed of the proposed method.