Physical inactivity is a major modifiable risk factor for morbidity, disability and premature mortality worldwide. This study assessed the geographical variations in the impact of environmental quality on physical inactivity among U.S. adults. Data on county-level prevalence of leisure-time physical inactivity came from the Behavioral Risk Factor Surveillance System. County environment was measured by the Environmental Quality Index (EQI), a comprehensive index of environmental conditions that affect human health. The overall EQI consists of five subdomains—air, water, land, social, and built environment. Geographically weighted regressions (GWRs) were performed to estimate and map county-specific impact of overall EQI and its five subdomains on physical inactivity prevalence. The prevalence of leisure-time physical inactivity among U.S. counties was 25% in 2005. On average, one standard deviation decrease in the overall EQI was associated with an increase in county-level prevalence of leisure-time physical inactivity by nearly 1%. However, substantial geographical variations in the estimated environmental determinants of physical inactivity were present. The estimated changes of county-level prevalence of leisure-time physical inactivity resulted from one standard deviation decrease of the overall EQI ranged from an increase of over 3% to a decrease of nearly 2% across U.S. counties. Analogous, the estimated changes of county-level prevalence of leisure-time physical inactivity resulted from one standard deviation decrease of the EQI air, water, land, social, and built environment subdomains ranged from an increase of 2.6%, 1.5%, 2.9%, 3.3%, and 1.7% to a decrease of 2.9%, 1.4%, 2.4%, 2.4%, and 0.8% across U.S. counties, respectively. Given the substantial heterogeneities in the environmental determinants of physical inactivity, locally customized physical activity interventions are warranted to address the most concerning area-specific environmental issue.