Multiple object tracking (MOT) and multiple identity tracking (MIT) each measure the ability to track moving objects visually. While prior investigators have mainly compared athletes and non-athletes on MOT, MIT more closely resembles dynamic real-life environments. Here we compared the performance and gaze behavior of handball players with non-athletes on both MOT and MIT. Since previous researchers have shown that MOT and MIT engage different eye movement strategies, we had participants track 3–5 targets among 10 moving objects. In MOT, the objects were identical, while in MIT they differed in shape and color. Although we observed no group differences for tracking accuracy, the eye movements of athletes were more target-oriented than those of non-athletes. We concluded that tasks and stimuli intended by researchers to demonstrate that athletes’ show better object tracking than non-athletes should be specific to the athletes’ type of sport and should use more perception-action coupled measures. An implication of this conclusion is that the differences in object tracking skills between athletes and non-athletes is highly specific to the skills demanded by the athletes’ sport.