The design of active and low-cost electrocatalyst for hydrogen evolution reaction (HER) is the key to achieving a clean hydrogen energy infrastructure. The most successful design principle of hydrogen electrocatalyst is the activity volcano plot, which is based on Sabatier principle and has been used to understand the exceptional activity of noble metal and design of metal alloy catalysts. However, this application of volcano plot in designing single-atom electrocatalysts (SAEs) on nitrogen doped graphene (TM/N4C catalysts) for HER has been less successful due to the nonmetallic nature of the single metal atom site. Herein, by performing ab initio molecular dynamics simulations and free energy calculations on a series of SAEs systems (TM/N4C with TM = 3d, 4d, or 5d metals), we find that the strong charge-dipole interaction between the negatively charged *H intermediate and the interfacial H2O molecules could alter the transition path of the acidic Volmer reaction and dramatically raise its kinetic barrier, despite its favorable adsorption free energy. Such kinetic hindrance is also experimentally confirmed by electrochemical measurements. By combining the hydrogen adsorption free energy and the physics of competing interfacial interactions, we propose a unifying design principle for engineering the SAEs used for hydrogen energy conversion, which incorporates both thermodynamic and kinetic considerations and allows going beyond the activity volcano model.