Abstract Human immunity influences the evolution and impact of novel influenza strains. Because individuals are infected with multiple influenza strains during their lifetime and each virus can generate a cross-reactive antibody response, it is challenging to quantify the processes that shape observed immune responses, or to reliably detect recent infection from serological samples. Using a Bayesian model of antibody dynamics at multiple timescales, we explain complex cross-reactive antibody landscapes by inferring participants’ histories of infection with serological data from cross-sectional and longitudinal studies of influenza A/H3N2 in southern China and Vietnam. We show an individual’s influenza antibody profile can be explained by a short-lived, broadly cross-reactive response that decays within a year to leave a smaller long-term response acting against a narrower range of strains. We also demonstrate that accounting for dynamic immune responses can provide a more accurate alternative to traditional definitions seroconversion for the estimation of infection attack rates. Our work provides a general model for explaining mechanisms of influenza immunity acting at multiple timescales based on contemporary serological data, and suggests a two-armed immune response to influenza infection consistent with competitive dynamics between B cell populations. This approach to analysing multiple timescales for antigenic responses could also be applied to other multi-strain pathogens such as dengue and related flaviviruses.