Identifying the correlates of exercise behavior is essential to combating physical inactivity as a public health concern. The purpose of this study is to identify the correlates of physical activity among urban Chinese adults under the social-ecological theoretical model in order to facilitate targeted interventions to promote physical activity. Using the socio-ecological model, we conducted a questionnaire survey among 1459 urban residents in Zhejiang and Shaanxi provinces of China, collecting data on individual demographic factors, sociological factors, environmental perception, and exercise behavior. Binary logistic regression models were employed to analyze the relationships between exercise behavior and socio-ecological factors. Male gender (p < 0.01), advanced age (p < 0.001), higher education level (p < 0.05), living independently from parents (p < 0.05), absence of childcare responsibilities (p < 0.01), residence in a county/prefecture-level city (p < 0.001), favorable neighborhood esthetics (p < 0.001), available greenways/parks (p < 0.001), and family support for exercise participation (p < 0.05) were significantly correlated with an increased likelihood of participating in physical activity. Male gender (p < 0.001), advanced age (p < 0.001), absence of childcare responsibilities (p < 0.05), good neighborhood vegetation (p < 0.01), availability of free neighborhood exercise facilities (p < 0.05), and support from friends for exercise participation (p < 0.01) were significantly correlated with an increased likelihood of engaging in physical activity for more than 150 min per week. BMI, community air quality, traffic safety, public safety, and level of social development were not major correlates. To promote exercise behavior, more attention should be paid to individuals who are female, young, have lower levels of education, bear childcare responsibilities, or reside in provincial capitals in China. Improving the habitat environment and providing convenient and affordable facilities should also be considered. Furthermore, support from family and friends can positively reinforce exercise behavior.