Background Idiopathic pulmonary fibrosis (IPF) is a major global health problem. The prevalence of the disease appears to be increasing. There is no curative therapy for IPF except lung transplantation. Chinese herbal medicines (CHMs) are showing promise for treatment of IPF. However, their effectiveness and safety are still unclear and deserve further investigation. The aim of this systematic review is to access the efficacy and safety of CHMs in treating IPF. Methods The protocol of this review is registered at PROSPERO. We searched seven main databases for randomized clinical trials (RCTs) on CHMs for IPF from their inception to June 4, 2018. The methodological quality of RCTs was assessed using the Cochrane risk of bias tool. All trials included were analyzed according to the criteria of the Cochrane Handbook. Review Manager 5.3, R-3.5.2 software, and Grade pro GDT web solution were used for data synthesis and analysis. Results Thirteen randomized clinical trials enrolling 733 patients were included. All trials included had clear outcome indicators. The methodological quality of included trials was generally “poor.” Few trials reported methods of randomization. One trial on Xuefu-zhuyu capsule assessed rate of acute exacerbation and mortality after treatment for 72 weeks and found no statistically significant difference between two groups. This meta-analysis demonstrated a significant improvement in QOL of IPF patients when CHMs was applied or combined with conventional medicine treatment. 6MWT was significantly improved in IPF patients after using CHMs or combined with conventional medicine treatment. CHMs treatment also had a certain improvement in TLC and DLCO, but the effect on FVC was not significant. Besides, CHMs failed to provide benefits in terms of PaO2. The reported adverse events were not obvious and severe. Conclusions Some CHMs seem effective and safe as alternative remedies for patients with IPF, suggesting that further study of CHMs in the treatment of IPF is warranted. Although this systematic review suggests that CHMs may have positive effect on quality of life, 6-minute walk test distance, and lung function (TLC, DLOC%) and seem to be relatively safe during the course of treatment, the results must be treated with great caution because of the methodological flaws of the included trials. Long-term and high-quality trials are needed in the future to provide clear evidence for the use of CHMs.