Polycystic ovary syndrome (PCOS) is the leading cause of anovulatory infertility, often requiring ovarian stimulation in affected women attempting to conceive. Male partner semen quality and shared lifestyle factors can significantly impact reproductive outcomes. However, current international guidelines lack evidence-based recommendations on the necessity and timing of semen analysis for the fertility management of anovulatory PCOS women. In a retrospective case-control study, semen analysis results of male partners of 187 anovulatory PCOS women scheduled for ovarian stimulation were analyzed and compared to a control group of 76 male partners of women with bilateral tubal occlusion. The prevalence of semen analysis results with at least one parameter classified as "borderline" and "pathological" among male partners of women with PCOS eligible to undergo ovarian stimulation was 51.3% and 22.5%, compared to 44.7% and 13.2% in the control group, respectively (p = 0.027). In the PCOS group, male body mass index (odds ratio, OR 1.478, p < 0.001), and smoking status (OR 6.228, p < 0.001) were significant predictors of pathological sperm test results, while no association was observed with any female characteristics (p > 0.05). The high frequency of pathological sperm analysis results provides lacking epidemiological data on semen quality in this population, emphasizing the critical need for early male fertility evaluation prior to ovarian stimulation in PCOS women. Moreover, our findings indicate that the risk prediction for abnormal semen quality cannot be based on the female's data.