We sought to determine whether lower fertility related quality of life or depression in men of couples with unexplained infertility is associated with low total testosterone levels, abnormal semen quality or erectile dysfunction. This study is a secondary analysis of a large, multicenter, randomized controlled trial in couples with unexplained infertility. Male partners underwent baseline semen analysis with measurement of fasting total testosterone and gonadotropin. They also completed surveys, including the FertiQOL (Fertility Quality of Life), the PHQ-9 (Patient Health Questionnaire-9) and the IIEF (International Index of Erectile Function). The primary study outcomes were total testosterone with low total testosterone defined as less than 264 ng/dl, semen parameters and the IIEF score. We performed multivariable logistic regression analyses adjusted for patient age, race, body mass index, education, smoking, alcohol use, infertility duration and comorbidity. A total of 708 men with a mean ± SD age of 34.2 ± 5.6 were included in study. Of the men 59 (8.3%) had a PHQ-9 score of 5 or greater, which was consistent with depression, 99 (14.0%) had low total testosterone and 63 (9.0%) had mild or worse erectile dysfunction. Neither the FertiQOL score nor depression was associated with total testosterone or any semen parameter. The FertiQOL score was inversely associated with erectile dysfunction (for every 5-point score decline AOR 1.30, 95% CI 1.16-1.46). Depressed men were significantly more likely to have erectile dysfunction than nondepressed men (AOR 6.31, 95% CI 3.12-12.77). In men in couples with unexplained infertility lower fertility related quality of life and depression are strongly associated with erectile dysfunction. However, neither is associated with spermatogenesis or testosterone levels. Erectile dysfunction in infertile men merits longitudinal investigation in future studies.