Background: Previous studies and meta-analyses have explored the relationship among testosterone, muscle strength, and physical function, to the best of our knowledge, no meta-analysis has investigated the effects of testosterone replacement therapy (TRT) on subgroup of relatively hypogonadal older men. Objective: The aim of this study was to evaluate the effect of TRT in older men with low testosterone levels. Methods: PubMed, Embase, and Web of Science were systematically searched for articles published between January 1990 and April 2020. We included randomized controlled studies that investigated the effect of TRT and included older men (age >60 years) with relatively low testosterone levels. Studies were extracted following the PRISMA flowchart, and the included randomized controlled trials were evaluated using RoB 2.0. Our main outcome was muscle strength changes after TRT evaluated using a metaregression of confounding factors. Secondary outcomes included changes in physical performance and the risk ratio of adverse events. Random-effects meta-analyses of TRT on muscle strength and physical function were performed. Results: Thirteen studies with 2,043 patients were included. The mean age of subjects in various studies ranged from 65.9 years to 76 years. Transdermal testosterone dosages ranged from 5 to 10 g/day, while intramuscular options were 125 mg/week or 200 mg every 2 weeks. Oral testosterone supplementation was given at 160 mg/day in one study. Pooled meta‐analyses revealed greater muscle strength improvement after TRT compared with placebo (Hedges’ g = 0.21; 95% CI: = 0.15–0.28). Intramuscular administration of TRT had greater efficacy (Hedges’ g = 0.74; 95% CI: = 0.34–1.14) than transdermal and oral TRT (p < 0.001). A metaregression revealed that baseline serum total testosterone was associated with muscle strength improvement (β = −0.004, p = 0.002). The risk ratios of adverse events, including elevated prostate-specific antigen, acute coronary syndrome, and prostate cancer, were not significantly different. Conclusion: TRT improved muscle strength in older, relatively hypogonadal men. The effect was more pronounced in populations with lower baseline testosterone levels.