An attractive alternative for extending the scaling of Flash-type memories is to replace the conventional floating gate (a poly-Si layer) by laterally isolated floating nodes in the form of nanoparticles. This floating-gate concept has led to the emergence of the so-called nanocrystal (NC) memories which have the potential of operating at lower voltages and higher speeds compared to the conventional non-volatile memories (NVMs) without compromising the criterion of non-volatility. NC memories also offer other advantages like a better immunity to the crosstalk effect arising from the floating-gate coupling of closely spaced adjacent cells and an increased design flexibility from which quantum confinement phenomena can be judiciously exploited for enhanced memory functionality. Among the different technological routes explored in the last few years for generating nanocrystals in the gate insulator of MOS devices, two major techniques have been utilized, namely deposition in vacuum and ion beam synthesis. During the last decade, we have extensively explored the ultra-low-energy ion-beam-synthesis (ULE-IBS) technique to produce single planes of Si-NCs in very thin (5–10 nm) insulator layers. This review summarizes more than 10 years of research efforts we and other groups have dedicated to the fabrication of NCs memories using this original technique. By exploiting the flexibility of the ULE-IBS route, Si-NCs single-transistor NVM cells using “classical” gate oxides (SiO2) have been successfully realized. More recently, advanced dielectric stacks employing high-κ dielectric (HfO2) as tunnel oxide and SiN as control oxide and host matrix for Si and Ge-NCs, have been fabricated and show promising performance for low-voltage operating-NVM devices. Particular emphasis is placed herein on material science issues such as anomalous and controlled oxidation processes. While much research is still needed for making reliable and competitive NC NVMs, our systematic investigations based on the ULE-IBS option demonstrate that a tight control and understanding of the properties of NCs memory cells can be achieved, provided that deep structural characterization is coupled to electrical studies.