This comprehensive study provides a detailed review toward ongoing research on emerging non-volatile memory technologies based nano-oscillators, i.e., from the perspective of materials to applications. Depending on the materials used to fabricate them, the whole class of emerging nano-oscillators has been broadly classified into two categories: (i) electronic and (ii) spintronic oscillators. Moreover, various material-compositions explored for fabricating the oscillatory devices, their physical properties responsible for generating the oscillations, and device architectures are thoroughly reviewed. Furthermore, various advanced computing applications (i.e., realized through these oscillatory devices), such as Internet of Things, neuro-inspired computing, and sensing, are also studied and their key attributes are highlighted.