Zinc (Zn) is a bioabsorbable metal that shows great potential as an implant material for orthopedic applications. Suitable concentrations of zinc ions promote osteogenesis, while excess zinc ions cause apoptosis. As a result, the conflicting impacts of Zn2+ concentration on osteogenesis could prove to be significant problems for the creation of novel materials. This study thoroughly examined the cell viability, proliferation, and osteogenic differentiation of rat bone marrow mesenchymal stem cells (rBMSCs) cultured in various concentrations of Zn2+ in vitro and validated the osteogenesis effects of zinc implantation in vivo. The effective promotion of cell survival, proliferation, migration, and osteogenic differentiation of bone marrow mesenchymal stem cell (BMSCs) may be achieved at a low concentration of Zn2+ (125 μM). The excessively high concentration of zinc ions (>250 μM) not only reduces BMSCs' viability and proliferation but also causes them to suffer apoptosis due to the disturbed zinc homeostasis and excessive Zn2+. Moreover, transcriptome sequencing was used to examine the underlying mechanisms of zinc-induced osteogenic differentiation with particular attention paid to the PI3K-AKT and TGF-β pathways. The present investigation elucidated the dual impacts of Zn2+ microenvironments on the osteogenic characteristics of rBMSCs and the associated processes and might offer significant insights for refining the blueprint for zinc-based biomaterials.