As a globally important energy-using country, China's energy efficiency improvement is crucial to achieving its energy-saving goals and economic transformation. This paper explores the improvement path, the improvement potential and the dynamic evolution of regional energy efficiency in China from 2005 to 2016 with an improved multidirectional efficiency analysis and the Markov model. The results indicate that (1) China's provincial energy efficiency is olive-shaped, and there is a significant spatial imbalance. (2) Most central provinces and a small number of eastern and western provinces need to adopt a one-way breakthrough path for their weak links. Most western provinces should adopt a step-by-step progressive or leap-forward path to improve energy efficiency. (3) The energy saving potential of the eastern and western regions is relatively large, and the potential value of CO2 emission reduction in the central region are relatively large. (4) In the short run, the comprehensive energy efficiency of different provinces aren't highly fluid between different levels; In the long run, the equilibrium state of energy efficiency in China will be at a mid-low level. Accordingly, it is recommended to strengthen exchanges between different regions, give full play to the resource advantages of each region, and differentiated and targeted energy efficiency policies should be carried out in different provinces.