Infertility affects 15% of men and contributes to nearly half of all cases of infertility. Infertile men usually have impaired spermatogenesis, presenting as azoospermia or various degrees of asthenospermia and oligozoospermia. Spermatogenesis is a complex and coordinated process, which is under precise modulation by the hypothalamic-pituitary-gonadal (HPG) axis. An aberrant hormone profile, especially an imbalance between testosterone (T) and estradiol (E2), plays an essential role in male infertility. In the male, E2 is produced mainly from the conversion of T by the aromatase enzyme. Theoretically, reducing an abnormally elevated T:E2 ratio using aromatase inhibitors (AIs) could restore the balance between T and E2 and optimize the HPG axis to support spermatogenesis. For decades, AIs have been used to treat male infertility empirically. However, owing to the lack of large-scale randomized controlled studies and basic research, the treatment efficacy and safety of AIs in male infertility remain controversial. Therefore, there is a need to summarize the clinical trials and relevant basic research on the application of AIs in the treatment of male infertility. In this narrative review, we summarized the application of AIs in the treatment of male infertility, including the pharmacological mechanisms involved, clinical trials focused on patients with different types of infertility, factors affecting treatment efficacy and the side-effects. A literature search was performed using MEDLINE/PubMed and EMBASE, focusing on publications in the past four decades concerning the use of AIs for treating male infertility. The search terms included AI, male infertility, letrozole, anastrozole, testolactone, azoospermia, oligozoospermia, aromatase polymorphisms, obesity and antiestrogens, in various combinations. Clinical studies demonstrate that AIs, especially nonsteroidal letrozole and anastrozole, could significantly inhibit the production of E2 and its negative feedback on the HPG axis, resulting in increased T and FSH production as well as improved semen parameters in infertile men. Large-scale surveys suggest that obesity may result in symptoms of hypogonadism in both fertile and infertile males, such as decreased semen quality and attenuated sexual function, which can be improved by AIs treatment. Polymorphisms of the aromatase gene CYP19A1, including single nucleotide polymorphisms and tetranucleotide TTTA repeats polymorphism (TTTAn), also influence hormone profiles, semen quality and treatment efficacy of AIs in male hypogonadotropic hypogonadism and infertility. The side-effects of AIs in treating male infertility are various, but most are mild and well tolerated. The application of AIs in treating male infertility has been off-label and empirical for decades. This narrative review has summarized the target patients, dose, treatment duration and side-effects of AIs. Polymorphisms of CYP19A1 that may affect AIs treatment efficacy were also summarized, but a full understanding of the mechanisms involved in AIs action requires further study.