Numerical knowledge, including number concepts and arithmetic procedures, seems to be a clear-cut case for abstract symbol manipulation. Yet, evidence from perceptual and motor behaviour reveals that natural number knowledge and simple arithmetic also remain closely associated with modal experiences. Following a review of behavioural, animal and neuroscience studies of number processing, we propose a revised understanding of psychological number concepts as grounded in physical constraints, embodied in experience and situated through task-specific intentions. The idea that number concepts occupy a range of positions on the continuum between abstract and modal conceptual knowledge also accounts for systematic heuristics and biases in mental arithmetic, thus inviting psycho-logical approaches to the study of the mathematical mind.This article is part of the theme issue ‘Varieties of abstract concepts: development, use and representation in the brain’.