The two known binary millisecond pulsars1,2 PSR1953 + 29 and PSR1855 + 09 with periods P =6.1ms and 5.4ms, respectively, and the long orbital-period binary radio pulsars3 are believed to have evolved from low-mass X-ray binaries (LMXBs)4. The neutron stars in these binaries are believed to have formed from the accretion-induced collapse of white dwarfs with shorter orbital periods5. These neutron stars have weak magnetic fields (3×108 G≲B≲6×1011 G) and rapid spin (5 ms≲P≲1s), compared with typical isolated pulsars, and are believed to have been spun up by accretion after their magnetic fields had decayed (see ref. 6 for review) on a timescale tD ˜9×106 yr (ref. 7). If correct, the fraction of pulsars which are binary millisecond pulsars is too large8 by a factor of >100. This severe discrepancy is removed if, as we propose here, the magnetic fields of neutron stars do not decay either in such binaries or in general. We also show that, if such neutron stars are formed from the accretion-induced magnetic flux and angular momentum-conserving collapse of white dwarfs, most of them are likely to have been born, and remain, spinning rapidly and to have weak magnetic fields, in agreement with observations of binary millisecond pulsars and LMXBs.