Bone diseases/injuries have been driving an urgent quest for bone substitutes for bone regeneration. Nanoscaled materials with bone-mimicking characteristics may create suitable microenvironments to guide effective bone regeneration. In this review, the natural hierarchical architecture of bone and its regeneration mechanisms are elucidated. Recent progress in the development of nanomaterials which can promote bone regeneration through bone-healing mimicry (e.g., compositional, nanocrystal formation, structural, and growth factor-related mimicking) is summarized. The nanoeffects of nanomaterials on the regulation of bone-related biological functions are highlighted. How to prepare nanomaterials with combinative bone-biomimicry features according to the bone healing process is prospected in order to achieve rapid bone regeneration in situ.