Considering both regional heterogeneity and industrial heterogeneity, this paper proposes a three-hierarchy meta-frontier data envelopment analysis (DEA) to decompose energy efficiency and energy-saving potential into three components, i.e., industrial structure, regional balance development, and management. Based on this approach, the paper then analyses the historical evolution and current status of energy efficiency in China. The results show the following. (1) From 2001 to 2015, Mainland China's energy efficiency increased by a factor of 2.48, primarily because of technological progress and the optimization of its industrial structure. (2) Nonetheless, the current energy efficiency in Mainland China remains very low and energy inefficiency is as high as 0.5922, indicating that 59.22% of total energy consumption is invalidly used. Severe industrial structure inefficiency and regional balance development inefficiency are the two main factors responsible for the very low energy efficiency. (3) By promoting industrial restructuring, regional balance development, and management improvement, Mainland China is expected to save, respectively, 13.67%, 24.54%, and 61.73% of its current total invalid energy consumption. Based on the estimation results, this paper formulates strategies to improve energy efficiency and save energy for China's provinces based on their specific situations, considering not only the direction of industrial restructuring in both the short and long terms but also the path to improved energy efficiency. In addition, a brief discussion about the applicability and the preconditions for an application of the multi-hierarchy meta-frontier approach is given at the end of the paper.