It is commonly believed that global patterns of motion in flocks, schools and crowds emerge from local interactions between individuals, through a process of self-organization. The key to explaining such collective behaviour thus lies in deciphering these local interactions. We take an experiment-driven approach to modelling collective motion in human crowds. Previously, we observed that a pedestrian aligns their velocity vector (speed and heading direction) with that of a neighbour. Here we investigate the neighbourhood of interaction in a crowd: which neighbours influence a pedestrian's behaviour, how this depends on neighbour position, and how the influences of multiple neighbours are combined. In three experiments, a participant walked in a virtual crowd whose speed and heading were manipulated. We find that neighbour influence is linearly combined and decreases with distance, but not with lateral position (eccentricity). We model the neighbourhood as (i) a circularly symmetric region with (ii) a weighted average of neighbours, (iii) a uni-directional influence, and (iv) weights that decay exponentially to zero by 5 m. The model reproduces the experimental data and predicts individual trajectories in observational data on a human ‘swarm’. The results yield the first bottom-up model of collective crowd motion.