Neighborhood compactness has been identified as a sustainable approach for fostering social relationships. However, existing studies have not clearly demonstrated whether neighborhood compactness in high-density cities directly influences the mental distress of community-dwelling older adults, or whether it indirectly influences mental distress through perceived accessibility to urban amenities (PA). Using a composite score of three dimensions (depressive symptoms, loneliness, and social isolation) to measure mental distress, and applying a mediation model within a cross-sectional design involving 947 older adults (aged ≥ 60) with at least mild symptoms in Hong Kong, our findings suggest an indirect pathway between neighborhood compactness, PA, and mental distress. Neighborhood compactness was positively associated with PA (B = 7.791, p < 0.001), and PA was negatively associated with mental distress (B = −0.12, p = 0.017). However, no direct impact of neighborhood compactness on mental distress was found. Moderated mediation analysis further indicated that neighborhood compactness and PA supported older females but not males. These results align with our hypothesis that (1) neighborhood compactness may not always be beneficial for older adults with mental distress in high-density cities unless compactness itself enhances PA and facility usage to support community-dwelling individuals, and (2) gender differences may result in varying interactions and perceptions of urban amenities and the built environment. To support the concept of “aging in place” in the future, urban plans aimed at enhancing neighborhood compactness and its social impacts should focus on addressing social inequality, including strategies to improve urban design, social participation, and gender-specific protocols, so that older adults can achieve better community awareness, environmental satisfaction, and facility usage, ultimately reducing mental distress.