This study analyzed whether gray matter volume (GMV) differences exist between the sexes in patients with major depressive disorder (MDD) and explored the relationships between these differences and neurotransmitter systems. This study enrolled 190 first-episode drug-naive patients with MDD and 293 healthy controls. All participants underwent T1-weighted high-resolution MRI. The interaction between the diagnosis (healthy controls vs. MDD) and sex (male vs. female) regarding GMV alterations was analyzed. The JuSpace toolbox, which covers a wide range of neurotransmitter systems, was used to identify the relationship between MDD-induced and sex-induced GMV alterations and specific receptor/transporter proteins in the brain. Sex-specific GMV differences were observed in the healthy controls but not in MDD patients. Male healthy controls had a larger GMV in the bilateral parahippocampal, lingual, inferior occipital, fusiform, cerebellar subregions, and left inferior temporal than female healthy controls, but several subregions of the thalamus had a larger GMV in female healthy controls than in male healthy controls. Sex-induced GMV alterations were associated with 5-hydroxytryptamine receptor subtype 1a, cannabinoid receptor, and dopamine receptor (P < 0.01, false discovery rate corrected). GMV differences were not detected in the main effect of diagnosis and the interaction of diagnosis and sex. Sex-specific GMV differences are associated with the spatial distribution of serotonin, dopamine, and cannabinoid neurotransmitter receptor systems. Sex-based physiological differences in the GMV may account for male and female susceptibility to and differences in the clinical symptoms of MDD.