Short-term memory is implicated in a range of cognitive abilities and is critical for understanding primate cognitive evolution. To investigate the effects of phylogeny, ecology and sociality on short-term memory, we tested the largest and most diverse primate sample to date (421 non-human primates across 41 species) in an experimental delayed-response task. Our results confirm previous findings that longer delays decrease memory performance across species and taxa. Our analyses demonstrate a considerable contribution of phylogeny over ecological and social factors on the distribution of short-term memory performance in primates; closely related species had more similar short-term memory abilities. Overall, individuals in the branch of Hominoidea performed better compared to Cercopithecoidea, who in turn performed above Platyrrhini and Strepsirrhini. Interdependencies between phylogeny and socioecology of a given species presented an obstacle to disentangling the effects of each of these factors on the evolution of short-term memory capacity. However, this study offers an important step forward in understanding the interspecies and individual variation in short-term memory ability by providing the first phylogenetic reconstruction of this trait’s evolutionary history. The dataset constitutes a unique resource for studying the evolution of primate cognition and the role of short-term memory in other cognitive abilities.