Bone regeneration repairs bone tissue lost due to trauma, fractures, and tumors, or absent due to congenital disorders. The extracellular matrix (ECM) is an intricate dynamic bio-environment with precisely regulated mechanical and biochemical properties. In bone, ECMs are involved in regulating cell adhesion, proliferation, and responses to growth factors, differentiation, and ultimately, the functional characteristics of the mature bone. Bone ECM can induce the production of new bone by osteoblast-lineage cells, such as MSCs, osteoblasts, and osteocytes and the absorption of bone by osteoclasts. With the rapid development of bone regenerative medicine, the osteoinductive, osteoconductive, and osteogenic potential of ECM-based scaffolds has attracted increasing attention. ECM-based scaffolds for bone tissue engineering can be divided into two types, that is, ECM-modified biomaterial scaffold and decellularized ECM scaffold. Tissue engineering strategies that utilize the functional ECM are superior at guiding the formation of specific tissues at the implantation site. In this review, we provide an overview of the function of various types of bone ECMs in bone tissue and their regulation roles in the behaviors of osteoblast-lineage cells and osteoclasts. We also summarize the application of bone ECM in bone repair and regeneration. A better understanding of the role of bone ECM in guiding cellular behavior and tissue function is essential for its future applications in bone repair and regenerative medicine.