Previous studies indicate that pollution exposure can increase risks of adverse birth outcomes, but Black communities are underrepresented in this research, and the potential moderating role of neighborhood context has not been explored. These issues are especially relevant in Louisiana, which has a high proportion of Black residents, an entrenched history of structural racism, the most pounds of toxic industrial emissions annually, and among the nation’s highest rates of low birthweight (LBW), preterm birth (PTB), and infant mortality. We investigated whether air pollution and social polarization by race and income (measured via the index of concentration at the extremes [ICE]) were associated with LBW and PTB among Louisiana census tracts (n = 1101) using spatial lag models. Data sources included 2011–2020 birth records, U.S. Census Bureau 2017 demographic data, and 2017 respiratory hazard (RH) from the U.S. Environmental Protection Agency. Both RH and ICE were associated with LBW (z = 4.4, P < 0.0001; z = −27.0, P < 0.0001) and PTB (z = 2.3, P = 0.019; z = −16.7, P < 0.0001), with no interaction. Severely polluted tracts had 36% higher and 25% higher risks of LBW and PTB, respectively, versus unpolluted tracts. On average, 2166 low birthweight and 3583 preterm births annually were attributable to pollution exposure. Tracts with concentrated social deprivation (i.e. low ICE scores) had 53% higher and 34% higher risks of LBW and PTB, respectively, versus intermediate or mixed tracts. On average, 1171 low birthweight and 1739 preterm births annually were attributable to concentrated deprivation. Our ecological study found that a majority of adverse birth outcomes in Louisiana (i.e. 67% of LBW and PTB combined) are linked to air pollution exposure or disadvantage resulting from social polarization. These findings can inform research, policy, and advocacy to improve health equity in marginalized communities.