Abstract Study question What are the impacts of different male factors on cumulative live birth and neonatal outcomes of IVF/ICSI cycles? Summary answer Severe male infertility factors may be negatively associated with fertilization and egg utilization, embryo development and cumulative live birth rate, but not with neonatal outcomes. What is known already Several studies have explored the effect of semen quality on IVF/ICSI clinical outcomes and have drawn contradictory conclusions because the semen quality of male partners could be fluctuated by many factors and there are many confounding factors from women partner. However, an all-round and definitive conclusion has not been reached yet, and clinicians have not yet determined whether men with abnormal semen parameters should be treated to improve their semen parameters and IVF clinical outcomes. Study design, size, duration The retrospective cohort study involved 6,580 males with abnormal sperm parameters and 17,254 males with normozoospermia who underwent their first IVF/ICSI cycle between January 2018 and September 2022 in the reproductive medicine center at a university hospital. They were divided into five different study groups: normozoospermia (N), moderate male factor (MMF), severe oligoasthenoteratozoospermia (OAT-S), azoospermia-husband (azoospermia-H), and azoospermia-donor (azoospermia-D). We compared fertilization, embryo development, cumulative live birth and neonatal outcome. Participants/materials, setting, methods Reproductive and neonatal outcomes of men with abnormal sperm parameters (MMF, OAT-S, azoospermia-H, and azoospermia-D) were studied through four propensity score matching (PSM) comparisons along with corresponding control groups(N) (match factors: including female age, BMI, Gn time, controlled ovarian stimulation protocol, number of oocytes obtained, endometrial thickness and duration of infertility). Fertilization outcomes were also compared stratified by IVF or ICSI. Main results and the role of chance The median ages of the females in the azoospermia, OAT-S, MMF, and N groups were 29, 30, 32 and 32 years old, respectively, which were comparable among groups after PSM. The fertilization rates were significantly reduced in OAT-S and azoospermia-H compared with N in ICSI cycles (67.05% and 65.05% versus 71.50%, p < 0.001, respectively). The oocyte utilization rates were also significantly declined in OAT-S and azoospermia-H compared with N in IVF/ICSI cycles (28.39% and 29.64% versus 34.83%, p < 0.001, respectively). Azoospermia-H group was divided into obstructive azoospermia (OA) and non-obstructive azoospermia (NOA). The conservative cumulative live birth rate (CLBR) in the OAT-S group with poor spermatogenic function decreased (63.6% vs. 72.9%, P < 0.001), and was also declined in the NOA group with the poorest spermatogenic function (62.4% vs. 72.9%, P < 0.001), compared with that in the OA group with mostly normal spermatogenic function. After binary multifactorial logistic regression, we found that conservative CLBR was lower in the OAT-S (aOR 0.675, 95%CI 0.536-0.850, P = 0.001) and NOA (aOR 0.574, 95%CI 0.384-0.857, P = 0.007) groups than in the OA group. No impact of sperm factor on obstetrical/perinatal outcomes was observed. In IVF/ICSI cycles, reproductive and neonatal outcomes were similar between MMF, azoospermia-D and normal semen groups. Limitations, reasons for caution The main limitation of this study is that it was a single-center, observational and retrospective study. However, the large sample size, use of PSMs and use of a multivariate regression model for a wide array of possible confounding factors rendered the conclusion relatively reliable. Wider implications of the findings This is the first study involving long-term outcomes (obstetrical and neonatal outcomes, cumulative pregnancy and live rates) from IVF/ICSI cycles for different male factors, which can help clinicians identify paternal influence in an all-round way. Trial registration number not applicable