The construction of highly active and durable electrocatalysts for hydrogen production from water electrolysis in a wide pH range is crucial for the sustainable development of clean energy. In this work, low-loading and ultrasmall Ir nanoparticles decorated on the surface of Ni/nitrogen-doped carbon nanofibers (NCNFs) are rationally constructed through a simple chemical reduction deposition route. Benefiting from the rich electrochemical active sites and excellent electrical conductivity, the obtained Ni-NCNFs-Ir catalyst exhibits superior hydrogen evolution reaction (HER) activities in both acidic and alkaline solutions. Specifically, the optimized 0.1Ni-NCNFs-5Ir catalyst with Ir content of 6.0 wt% exhibits remarkable low overpotentials of 22 and 25 mV at 10 mA cm−2 in 0.5 M H2SO4 and 1 M KOH, respectively, which are better than many other reported Ir- or Ni-based electrocatalysts. In addition, the prepared 0.1Ni-NCNFs-5Ir catalyst shows favorable durability. Our theoretical results reveal that the relevant C- and Ir-sites jointly contribute to the high HER catalytic property of the Ni-NCNFs-Ir system, and two complicated electron transfer processes (Ni → C → N and Ir → C → N) are beneficial to promote the electrocatalytic performance. This work provides a meaningful strategy to develop high-efficiency HER electrocatalysts in a wide pH range.