Osteoporosis is the most common bone metabolic disease, and it is becoming increasingly common as the global population ages. Osteoporosis and its complications, such as fractures and pain, negatively affect patient quality of life and easily lead to disability, placing enormous burdens on society. Although several anti-osteoporosis drugs are currently available, many adverse reactions have been observed during the long-term application of these drugs. Therefore, safer and more useful medications are urgently needed to replace those currently available. Chinese herbal medicine has been extensively used to treat osteoporosis, and the current literature confirms that such medicines have anti-osteoporosis effects, are safe, and have minimal side effects. Thus, Chinese herbal medicines are natural alternatives to pharmaceutical approaches to treating osteoporosis, and these medicines must be further developed and utilized. In this article, we review the mechanisms underlying the anti-osteoporosis effects of single herbal extracts and traditional Chinese medicine (TCM) formulas that have been elucidated since 2013, providing key evidence and support for future research on the anti-osteoporosis effects of Chinese herbal medicines. In addition, due to the complexity of the ingredients in Chinese herbal medicine, more thorough investigations are needed to determine the specific ingredients that are effective in osteoporosis treatment. Therefore, identifying the effective ingredients of Chinese herbal medicines will be a necessary focus in laboratory research and clinical application.