Background: The objective of this study was to examine the correlation between low muscle mass (LMM) and depression, with a specific focus on identifying the sex-specific relationship between LMM and depression in a large sample. Methods: This population-based cross-sectional study involved 292,922 community-dwelling adults from 2012 to 2019. Measurements were taken using the Center for Epidemiological Studies Depression (CESD) scale and body composition analyses. Depression was defined as a CESD score ≥ 16, and severe depression as a CESD score ≥ 22. LMM was defined as an appendicular muscle mass/height2 below 7.0 kg/m2 in men and below 5.4 kg/m2 in women. Sex-based multivariable logistic regression analyzed the LMM–depression association, adjusting for confounders, with depression status and severe depression status as dependent variables. Results: Both men and women in the LMM group had an increased odds of depression (men, adjusted odds ratio = 1.13 [95% confidence interval = 1.03–1.12]; women, 1.07 [1.03–1.23]) and severe depression (men, 1.20 [1.05–1.36]; women, 1.10 [1.04–1.15]) compared to those in the control group. Men showed a stronger association between LMM and the presence of depression (p for interaction = 0.025) and the presence of severe depression (p for interaction = 0.025) compared to women. Conclusions: Decreased muscle mass was independently associated with increased chances of depression and severe depression in both sexes, with a significantly stronger association in men compared to women. This highlights the potential significance of LMM as a predictor of depression, particularly in men.