Object tracking has become an important branch of computer vision currently. In recent years, because the tracking algorithm with kernel correlation filter uses the properties of circulant matrix, the main operation is element-wise product in the frequency domain. It achieved the great performance and speed than the previous tracking algorithm. However, when the target size changes and the target is seriously blocked, it couldn’t track accurately. Based on the above reasons, a scale updating algorithm and acoarse-to-fine target relocation algorithm are proposed to improve the KCF algorithm. The algorithm is transplanted on the eight-core DSP processor TMS320C6678 successfully. Through multi-core parallel processing，it achieved 30 frames/s real-time tracking frame rate.