Estimating the impact of air quality on productivity poses significant challenges, primarily due to issues of endogeneity and the potential correlation between errors in the traditional two-step approach. This approach involves calculating productivity in the initial step and subsequently estimating the impact in the second step. To address these challenges, we employ an endogenous stochastic frontier analysis framework to identify the causal effect of air quality on energy efficiency in China. Our findings reveal a strong negative relationship between air quality and energy efficiency. A 10% increase in daily average PM2.5 concentrations results in a decrease in energy efficiency by 5%. The detrimental impact of air pollution on energy efficiency is particularly pronounced in cities characterized by outdated production technology or higher levels of market competition. Furthermore, our analysis highlights two primary mechanisms through which air pollution impedes the improvement of energy efficiency: by inhibiting innovation activities and constraining the development of the tertiary industry. Lastly, by assessing the potential improvement in air quality, we estimate a significant energy conservation potential of 1037.45–2192.79 million kilowatt-hours during the period from 2004 to 2017. On the condition that fossil energy still dominates the energy structure in current China, improved energy efficiency can lead to less emissions. The measures on reducing the air pollution can achieve much larger effects.