We aimed to identify how the COVID-19 lockdown affected changes, including the potential for longer term sustained changes, in physical activity, as compared to immediately prior to the pandemic. Physical activity's significant role in overall health is known to be influenced by the surrounding environment, such as one's neighborhood, prompting this study of physical activity and its relationship with individual-level and neighborhood-level factors within the pandemic timeline. A statewide online survey assessed adults' self-reported weekly minutes of moderate-to-vigorous physical activity (MVPA) comparing a typical week immediately prior to the pandemic and during the pandemic (prewidespread vaccination) using negative binomial models. Overall, MVPA decreased during the pandemic, though the decrease was driven largely by the reduction in MVPA outside one's neighborhood. In contrast, MVPA done within one's neighborhood increased over time. This change in MVPA done within one's neighborhood was not uniform across several characteristics including income level favoring those with the highest income (p < .05) and race/ethnicity favoring those self-reporting as non-Hispanic White (p < .05). While several factors, including higher Walk Scores, were associated with higher levels of MVPA without evidence of change over time, evidence of a differential effect over time was seen for other key indicators of social and structural determinants of health including income and race/ethnicity. This study can add to the existing literature surrounding not only COVID-19 but also neighborhood built environmental research seeking to identify factors associated with changes in MVPA, a known indicator of overall health and health-related outcomes.