The high cost of drilling deep wells is the main barrier to the widespread exploitation of deep geothermal energy. Percussive drilling is one of the significant drilling technologies used in energy exploration projects. However, there is no good quantitative understanding of how much energy in percussive drilling is consumed in pulverization, heating, the kinetic energy of particles, acoustic emission, etc. In this study, energy efficiency is quantitatively investigated to understand the percussive drilling process better. The dynamic percussive drilling was evaluated using a modified split Hopkinson pressure bar (SHPB) system and non-contact measurements. The amount of energy dissipated in different processes and the overall energy efficiency was estimated for Kuru granite, Balmoral granite, and Kivijärvi gabbro. The energy spent on the kinetic energy Ek of fragments was evaluated using a high-speed camera, whereas the energy consumed on heat or the thermal energy Et was obtained by high-speed infrared imaging. The cracking energy Ec was measured by using the surface energy of rock and the total newly created surface areas. The results indicate that the fragment size distribution of these three rocks generally varies with the penetration speed, and the fragmentation level of these rocks increases with the penetration speed. The input energy and the energy consumption grow with the increase of the penetration speed. The proportions of Et, Ek, and Ec in the total energy consumption for these three rocks increase with the penetration speed. The energy efficiency obtained from the dynamic indentation experiments for the three rocks generally increases with the penetration speed and almost approaches a limit value when the penetration speed is high. A model is improved to describe the relationship between energy efficiency and penetration speed quantitatively. Therefore, the penetration process should be optimized to balance the high drilling efficiency and the low energy consumption.