Bones support the body as part of the human musculoskeletal system. They also contain bone marrow, which is a site of hematopoiesis. Bone marrow mesenchymal stem cells play a vital role by regulating skeletal tissue formation and maintaining hematopoiesis. While the presence of bone marrow-derived mesenchymal stem cells has been indicated, they have yet to be fully understood in vivo. Recent studies using genetic mouse models revealed that perivascular stromal cells function as mesenchymal stem cells, and their differentiation status may vary during the early stage of life to adulthood. Furthermore, studies have investigated the underlying mechanisms that regulate the cell fate decision of mesenchymal stem cells. These findings could lead to the design of new therapeutic approaches for metabolic bone disease and hematopoietic disease.