Sperm experience intense and varied selection that dramatically impacts the evolution of sperm quality. Selection acts to ensure that sperm are fertilization-competent and able to overcome the many challenges experienced on their way towards eggs. However, simply being able to fertilize an egg is not enough to ensure male fertility in most species. Owing to the prevalence of female multiple mating throughout the animal kingdom, successful fertilization requires sperm to outcompete rival sperm. In addition, females can actively influence sperm quality, storage or utilization to influence male fertility. This review provides an overview of how these selective forces influence the evolution of sperm quality. After exploring the link between sperm traits and male fertility, we examine how post-mating competition between rival ejaculates influences the evolution of sperm quality. We then describe how complex genetic, social and sexual interactions influence sperm quality, focusing on the importance of seminal fluid and interactions between sperm and the female's reproductive tract. In light of the complexities of selection on sperm traits, greater use of multivariate approaches that incorporate male-male, sperm-sperm and sperm-female interactions to study sperm quality will enhance our understanding of how selection acts on sperm traits and factors influencing male fertility. Because the metric of male reproductive success--fertilization--is the same across the animal kingdom, we argue that information about sperm evolution gained from non-human animals has enormous potential to further our understanding of the factors that impact human fertility.