Quark matter with only $u$ and $d$ quarks ($ud$QM) might be the ground state of baryonic matter at large baryon number $A>A_{\rm min}$. With $A_{\rm min}\gtrsim 300$, this has no direct conflict with the stability of ordinary nuclei. An intriguing test of this scenario is to look for quantum nucleation of $ud$QM inside neutron stars due to their large baryon densities. In this paper, we study the transition rate of cold neutron stars to $ud$ quark stars ($ud$QSs) and the astrophysical implications, considering the relevant theoretical uncertainties and observational constraints. It turns out that a large portion of parameter space predicts an instantaneous transition, and so the observed neutron stars are mostly $ud$QSs. We find this possibility still viable under the recent gravitational wave and pulsar observations, although there are debates on its compatibility with some observations that involve complicated structure of quark matter. The tension could be partially relieved in the two-families scenario, where the high-mass stars ($M\gtrsim2 M_{\odot}$) are all $ud$QSs and the low-mass ones ($M\sim1.4\, M_{\odot}$) are mostly hadronic stars. In this case, the slow transition of the low-mass hadronic stars points to a very specific class of hadronic models with moderately stiff EOSs, and $ud$QM properties are also strongly constrained.

26 pages, 11 figures