Background: The injection of bone mesenchymal stem cells (BMSCs) for osteoarthritis (OA) treatment fails to address the disrupted extracellular microenvironment, limiting the differentiation and paracrine functions of BMSCs and resulting in suboptimal therapeutic outcomes. Proteoglycans (PGs) promote cell differentiation, tissue repair, and microenvironment remodeling. This study investigated the potential of combining PGs with BMSCs to increase the efficacy of OA treatment. Methods: We evaluated the effects of PG on BMSC and chondrocyte functions by adding various PG concentrations to the culture media. Additionally, a Transwell system was used to assess the impact of PG on the communication between BMSCs and chondrocytes. The results of the in vitro experiment were verified by tissue staining and immunohistochemistry following the treatment of OA model rats. Results: Our findings indicate that PG effectively induces Col II expression in BMSCs and enhances the paracrine secretion of TGF-β1, thereby activating the TGF-β signaling pathway in chondrocytes and increasing PRG4 gene expression. Compared with the other groups, the BMSC/PG treatment group presented a smoother articular surface and more robust extracellular matrix than the other groups in vivo, with significantly increased expression and distribution of Smad2/3 and PRG4. Conclusions: PG enhances BMSC differentiation into chondrocytes and stimulates paracrine TGF-β1 secretion. Proteoglycans not only promote chondrocyte differentiation and paracrine TGF-β1 signaling in BMSCs but also increase the sensitivity of chondrocytes to TGF-β1 secreted from BMSCs, leading to PRG4 expression through the TGFR/Smad2/3 pathway. Proteoglycans can enhance the therapeutic effect of BMSC treatment on OA and have the potential to delay the degeneration of OA cartilage.