This paper proposes an adaptive template block for object tracking in typical block matching algorithms. The bounding box of moving object would have different scales while the object moves from near to far or from far to near. Therefore, the typical block matching algorithms using the fixed template block would obtain inaccurate object tracking and object recognition. The proposed adaptive template block is used in block matching algorithm to obtain accurate and reliable tracking of object with different scales. Experimental results demonstrate that the typical block matching algorithm using proposed adaptive template block have better performance than one using the fixed template block in object tracking.