Environmental pollutants are claimed to be major factors involved in the progressive decline of the fertility rate worldwide. Exposure to the heavy metal Cadmium (Cd) has been associated with reproductive toxicity due to its ionic mimicry. However, the possible direct accumulation of Cd in human sperm cells has been poorly investigated. In this study, we aimed to clarify the possible direct effect of Cd exposure on sperm function through the analysis of its cell accumulation. Semen samples from 30 male subjects residing in high environmental impact areas and adhering to the "Exposoma e Plurifocalità nella Prevenzione Oncologica" campaign for testis cancer prevention were compared with semen samples from 15 males residing in low exposure areas. Semen levels and cell Cd content were quantified by inductively coupled plasma (ICP) spectroscopy. Cell Cd distribution was assessed by scanning electron microscopy coupled with energy dispersive spectroscopy (SEM-EDS). The impact of Cd on sperm function was evaluated by the in vitro exposure to the heavy metal, whilst possible scavenging approaches/agents were assessed. In addition to higher values of semen Cd, exposed subjects showed a reduction in total motile sperm fraction compared to not-exposed controls (59.6% ± 13.6% vs. 66.3% ± 7.3%, p = 0.037). Semen Cd levels were also significantly correlated with SEM-EDS signals of Cd detected on the head and neck of sperm (respectively p = 0.738, p < 0.001 and ρ = 0.465, p < 0.001). A total of 2h of in vitro exposure to 0.5μM Cd was associated with a significant reduction of sperm progressive motility. Scavenging approaches with either hypo-osmotic swelling or 10μM reduced glutathione were ineffective in blunting cell Cd and restoring motility. The reduction of exposure levels appears to be the main approach to reducing the reproductive issues associated with Cd.