When interacting with the world, people can dynamically split attention across multiple objects in the environment, both when the objects are stationary and when the objects are moving. This type of visual processing is commonly studied in lab settings using either static selection tasks or moving tracking tasks. We describe performance limits that are common to both tasks, including limits on capacity, crowding, visual hemifield arrangement, and speed. Because these shared limits on performance suggest common underlying mechanisms, we examine a set of models that might account for limits across both. We also review cognitive neuroscience data relevant to these limits, which can provide constraints on the set of models. Finally, we examine performance limits that are unique to tracking tasks, such as trajectory encoding, and identity encoding. We argue that a complete model of multiple object tracking must account for both those limits shared between static selection and dynamic tracking, as well as limits unique to tracking. It must also provide neurally plausible mechanisms for the underlying processing resources. WIREs Cogn Sci 2015, 6:109–118. doi: 10.1002/wcs.1328This article is categorized under: Psychology > Attention