China has implemented a variety of energy efficiency policies to achieve energy saving and carbon abatement targets. While the national carbon market has been launched and China has opted to give market-oriented policies a greater role in achieving net zero, energy efficiency policies still constitute a significant portion of the policy mix. However, the substantial drawback of energy efficiency policies compared to carbon trading, the rebound effect, has made energy efficiency policies debatable. It is crucial to precisely understand the magnitude of the rebound effect to design and implement effective energy policies. Our study estimates and compares the direct energy rebound and output expansion effects in China's manufacturing and service sectors using firm-level data from 2013 to 2016 and employing panel regression analysis. We also analyze the impact of firm characteristics such as ownership status, geographical location, and industry type on enterprises' rebound effects. We found a 33.5% direct rebound effect and a 19.3% output expansion effect in the manufacturing sector. In the service sector, the direct energy rebound effect was 57.9%, a larger one than the manufacturing sector. We also estimate direct energy rebound effects for key manufacturing and service enterprises. Our findings reveal that key manufacturing enterprises exhibit a larger rebound effect compared to the overall manufacturing sector, while key service enterprises show a smaller rebound effect than the overall service sector. Public, foreign, and heavy service enterprises show a smaller energy rebound effect, while northeastern manufacturing and service enterprises exhibit a larger rebound effect. We suggest that China's carbon market should also cover key service enterprises. For those below the threshold, enhancing energy management is essential to mitigate the rebound effect. Aligning with carbon emission cap targets, rather than relying solely on energy efficiency policies, could more effectively achieve energy savings and carbon abatement.