Nanocrystalline titanium nitride (TiN) has been determined to be a promising alternative to noble metal palladium (Pd) for fabricating base membranes for the energy-efficient production of pure hydrogen. However, the mechanism of transport of hydrogen through a TiN membrane remains unclear. In this study, we established an atomistic model of the transport of grain boundary hydride ions through such a membrane. High-resolution transmission electron microscopy and X-ray reflectivity confirmed that a nanocrystalline TiN1.0 membrane with a (100) preferred growth orientation retained about 4 Å-wide interfacial spaces along its grain boundaries. First-principles calculations based on the density functional theory showed that these grain boundaries allowed the diffusion of interfacial hydride ion defects with very small activation barriers (<12 kJ mol-1). This was substantiated by the experiment. In addition, the narrow boundary produced a sieving effect, resulting in a selective H permeation. Both the experimental and theoretical results confirmed that the granular microstructures with the 4 Å-wide interlayer enabled the transition metal nitride to exhibit pronounced hydrogen permeability.