Social distancing (e.g., maintaining two-meter distance between people, restricting group gatherings) is recommended to reduce the risk of COVID-19 transmission. Social distancing guidelines might not broadly consider the varying built environment contexts in which people live, however. Controlling for key covariates, we examined the association of several built environment characteristics with COVID-19 cases in New York City. This cross-sectional ecological study used ZIP code tabulation area (ZCTA) data from New York City. We found positive, significant adjusted associations between the COVID-19 case rate and average people per household, percentage of Black or African American population, percentage of Hispanic population, and percentage of the population over sixty-five years old. Population per square kilometer and percentage taking public transit to work were both negatively associated with case rates. Percentage of sidewalks was negatively associated with case rates, but parks were not significant. ZCTAs with higher percentage area of sidewalks were associated with lower case rates, controlling for potential confounders. Consideration of the local characteristics of the built environment could inform context-specific COVID-19 prevention guidelines, such as increasing street space for pedestrian use in high-density neighborhoods.