Many bivalve mollusks display remarkable sex differentiation of gonadal accumulation of manganese (Mn), but the underlying processes responsible for such differences have seldom been explored. In this study, the accumulation of Mn in male and female gonads during the reproductive cycle of oysters was first examined, and the distributions of Mn in oocytes and sperm cells at different developmental stages were imaged by the nanoscale secondary ion mass spectrometry (NanoSIMS) at the subcellular level. We found that the distribution and accumulation of Mn during oogenesis were closely associated with the formation and translocation of cortical granules. This is the first time that the enrichment of Mn was directly visualized in cortical granules, which was identified as the major storage site of Mn in oocytes of oysters. Yolk granules were revealed as another storage pool of Mn in oyster oocytes with lower accumulation. In contrast, Mn was mainly distributed in the nucleus of sperm cells with accumulation levels much lower than those in cortical and yolk granules of oocytes. These results demonstrated great differences of the subcellular localization and accumulation capacity of Mn between oocytes and sperm cells in oysters, implying the sex differentiation in susceptibility of reproductive response to Mn stress. Our study also highlights the importance of gender difference in future biomonitoring and ecotoxicological studies of Mn in marine bivalves.