Enhancing energy efficiency is globally regarded as an effective way to reduce carbon emissions. In recent years, the energy efficiency of China has gradually improved; however, energy consumption and CO2 emissions are still increasing. To better understand the reasons for this, we evaluated the energy rebound effect (RE) of 30 provinces in China over the period 2001–2017 by employing stochastic frontier analysis (SFA) and the system generalized method of moments (system-GMM) approach, and explored the extent to which the RE affects CO2 emissions. Asymmetric and regional heterogeneity analyses were also conducted. The results indicate that the national average RE was 90.47% in the short run, and 78.17% in the long run, during the sample period. Most of the provinces experienced a partial RE, with a backfire effect occurring in some provinces such as Guangxi and Henan. The RE was associated with significant increases in CO2 emissions; specifically, a 1% increase in the short-run RE led to an increase in CO2 emissions of approximately 0.818%, and a 1% increase in the long-run RE resulted in an increase in CO2 emissions of approximately 0.695%. Moreover, significant regional differences existed in the impact of the RE on CO2 emissions; in regions with high emissions and a high RE, the CO2 reduction effect from the marginal decline in the RE was much more pronounced than that in other regions.