Worldwide population aging is currently in acceleration, which is especially true for China. Echoing the advocacy of “active aging” and “age-friendly communities”, governments and researchers across the world are paying more attention to the impact of neighborhoods on the health of older adults. Using the Ecological Model of Aging, this study aimed to discuss the relationships between neighborhood environment, lifestyle, and health of older adults, and to compare the differences among older adults of different age groups. The results showed that landscape environment has a direct effect on the health of older adults, while leisure environment has an indirect effect through lifestyle. Both leisure environment and landscape environment directly encourage older adults to take part in outdoor activity, in which the former mainly promotes the social participation of the high-aged (aged 80+) group, while the latter merely promotes that of the middle-aged (aged 70–79) group. The positive effect of social participation on health is gradually strengthened with the increase of age. Meanwhile, outdoor activity has its greatest effect on the middle-aged (aged 70–79) group, but not the low-aged (aged 60–69) group. To effectively boost the health of older adults and promote active aging, adequate considerations should also be given to the differentiated demands of older adults of different age groups, optimization of neighborhood environment, as well as cultivation of an amicable atmosphere.