To identify differences in human immunodeficiency virus (HIV) risky behavior and healthy practices between rural and urban men who have sex with men (MSM) in Southeast China, a cross-sectional study was conducted on MSM aged ≥ 18 years recruited from four community-based organizations (CBOs) in seven cities in Zhejiang Province between October 2022 and March 2023. An electronic survey questionnaire was completed by the study participants to collect HIV risky behavior and healthy practices. The chi-square test and multiple logistic regression were used for analysis. Among the 1,993 MSM participants, 21.1% (420) were rural MSM. Compared with urban MSM, rural MSM was more likely to have a low level of education (high school and below; 46.2% vs. 40.2%, p = 0.028) and a low annual income (≤ 90,000 RMB; 71.2% vs. 64.7%, p = 0.013). Multiple logistic regression showed that, compared with urban MSM in the past 6 months behaviors, rural MSM had lower odds of finding casual sex partners on the Internet (adjusted odds ratio [aOR]: 0.790, 95% confidence interval [CI]: 0.633–0.986), using stimulants with online casual sex partners (aOR: 0.688, 95% CI: 0.506–0.936), and high odds of ever having sex with a female (aOR: 1.460, 95% CI:1.036–2.058). Rural MSM reported lower odds of an HIV knowledge score of 2–3 (aOR: 0.586, 95% CI: 0.454–0.756), HIV testing more than twice (aOR: 0.583, 95% CI: 0.455–0.748), and obtaining four to five types of HIV information (aOR: 0.661, 95% CI: 0.529–0.826), as well as higher odds of high/very high awareness of HIV infection risk (aOR: 2.312, 95% CI: 1.638–3.263), compared to urban MSM. Rural MSM and urban MSM reported discrepancies in HIV risky behavior, HIV knowledge and HIV-related healthy practices. Paying more attention to HIV risky behavior and improving healthy practices in rural areas may help to prevent HIV transmission.