Abstract Scale variations occur frequently and present a great challenge for crowd counting in practical applications. In this paper, we propose a scale adaptive network to address the scale variation problem for crowd counting. We design a scale expansion unit which uses normal and dilated convolution to expand the receptive field size range of its input and connect several such units densely to cover a large range of densely distributed receptive field sizes so as to fit objects of different sizes in images. To alleviate competition among different scales, especially the negative effect of inappropriate scales, we also design a residual channel-wise re-weighting unit which is inserted after each scale expansion unit to enhance informative feature channels. We evaluate the effectiveness of the proposed scale adaptive network on ShanghaiTech-B and WorldExpo’10 datasets.