The evolution of the adaptive immune system is characterized by changes in the relative abundances of the B and T-cell clones that make up its repertoires. To fully capture this evolution, we need to describe the complex dynamics of the response to pathogenic and self-antigenic stimulations, as well as the statistics of novel lymphocyte receptors introduced throughout life. Recent experiments, ranging from high-throughput immune repertoire sequencing to quantification of the response to specific antigens, can help us characterize the effective dynamics of the immune response. Here we describe mathematical models informed by experiments that lead to a picture of clonal competition in a highly stochastic context. We discuss how different types of competition, noise and selection shape the observed clone-size distributions, and contrast them with predictions of a neutral theory of clonal evolution. These mathematical models show that memory and effector immune repertoire evolution is far from neutral, and is driven by the history of the pathogenic environment, while naive repertoire dynamics are consistent with neutral theory and competition in a fixed antigenic environment. Lastly, we investigate the effect of long-term clonal selection on repertoire aging.