Inspired by Approximate Dynamic Programming (ADP) and the Algebraic Riccatic Equation (ARE), this paper investigate a new optimal tracking control strategy for a class of discrete-time linear quadratic regulation (LQR) problems with disturbance. First, the optimal tracking problem is converted into designing infinite-horizon optimal regulator for the tracking error dynamics via system transformation. Then we compute the optimal tracking control policy, which can be considered as a way to solve the ARE of the well-known discrete-time optimal control problem forward in time. The iterative ADP algorithm via Heuristic Dynamic Programming (HDP) technique is introduced to solve the value function of the controlled system. To verify its robustness, disturbance is added to the controlled system. The simulation results show the effectiveness and robustness of the proposed algorithm in this paper.