Different physical exercise modalities have been widely studied in patients having heart failure with preserved ejection fraction (HFpEF) but with variably reported findings. We, therefore, conducted a systematic review and meta-analysis to evaluate whether the efficacy of physical activity in the management of HFpEF is related to exercise modalities. PubMed and Embase were searched up to July 2021. The eligible studies included randomized controlled trials that identified effects of physical exercise on patients with HFpEF. Sixteen studies were included to evaluate the efficiency of physical exercise in HFpEF. A pooled analysis showed that exercise training significantly improved peak oxygen uptake (VO2), ventilatory anaerobic threshold, distance covered in the 6-minute walking test, the ratio of early diastolic mitral inflow to annular velocities, the Short Form 36 physical component score, and the Minnesota Living with Heart Failure Questionnaire total score. However, the changes in other echocardiographic parameters including the ratio of peak early to late diastolic mitral inflow velocities, early diastolic mitral annular velocity, and left atrial volume index were not significant. Both high-intensity and moderate-intensity training significantly improved exercise capacity (as defined by peak VO2), with moderate-intensity exercise having a superior effect. Furthermore, exercise-induced improvement in peak VO2 was partially correlated with exercise duration. Physical exercise could substantially improve exercise capacity, quality of life, and some indicators of cardiac diastolic function in patients with HFpEF. A protocol of moderate-intensity exercise training lasting a longer duration might be more beneficial compared with high-intensity training for patients with HFpEF.