Hypercholesterolemia due to a high-cholesterol diet is linked to numerous diseases and may lead to male infertility. However, the underlying mechanism remains unknown. The maintenance of male fertility requires intact testicular structures (including seminiferous tubules and mesenchyme) and functioning cells (Leydig cells, Sertoli cells and germ cells, etc.), production of appropriate concentrations of sex hormones, and cooperation among testicular cells. Thus, we considered whether male fertility declined as the structure and function of testicular cells were altered in rats on a high-cholesterol diet. Male Sprague Dawley rats were fed either a standard or a high-cholesterol diet for 16 weeks. Serum sex hormones, lipid components, semen quality, and fertility rate were assayed in the rats. The 3β-hydroxysteroid dehydrogenase (3β-HSD), Wilms tumor 1 (WT-1), and deleted in azoospermia-like (DAZL) were regarded as specific markers of Leydig, Sertoli, and germ cells in rats. In addition, the ultrastructure of the testis and expression levels of particular marker molecules of testicular cells were further investigated. Compared to rats fed on a regular diet, the serum testosterone levels and sperm progressive motility decreased in rats fed high cholesterol. Moreover, we observed a deformed nucleus, dilated smooth endoplasmic reticulum, and swollen mitochondria of Leydig cells and a schizolytic nucleus of Sertoli cells in rats on a high-cholesterol diet. The 3β-HSD, WT-1, and DAZL protein expression levels were significantly reduced in rats on a high-cholesterol diet. Our results showed that a high-cholesterol diet adversely affected testosterone production and sperm progressive motility, possibly due to Leydig, Sertoli, and germ cell abnormalities.