The paucity of old isolated accreting neutron stars in ROSAT observations is used to derive a lower limit on the mean velocity of neutron stars at birth. The secular evolution of the population is simulated following the paths of a statistical sample of stars for different values of the initial kick velocity, drawn from an isotropic Gaussian distribution with mean velocity $0\leq < V>\leq 550$ ${\rm km s^{-1}}$. The spin--down, induced by dipole losses and the interaction with the ambient medium, is tracked together with the dynamical evolution in the Galactic potential, allowing for the determination of the fraction of stars which are, at present, in each of the four possible stages: Ejector, Propeller, Accretor, and Georotator. Taking from the ROSAT All Sky Survey an upper limit of $\sim 10$ accreting neutron stars within $\sim 140$ pc from the Sun, we infer a lower bound for the mean kick velocity, $ < V>\gtrsim 200-300$ ${\rm km s^{-1}},$ corresponding to a velocity dispersion $��_V\gtrsim 125-190$ km s$^{-1}$. The same conclusion is reached for both a constant magnetic field ($B\sim 10^{12}$ G) and a magnetic field decaying exponentially with a timescale $\sim 10^9$ yr. Such high velocities are consistent with those derived from radio pulsar observations. Present results, moreover, constrain the fraction of low velocity stars, which could have escaped pulsar statistics, to less than 1%.

13 pages, 6 PostScript figures, accepted to ApJ