For hydrogen purification at low temperature, multi-layered hydrogen-permeable thin films with amorphous Ni64Zr36/M/Ni64Zr36 (M = V, Nb) structure were developed and the microstructure and hydrogen permeation performance were examined. The multi-layered thin films were deposited through DC-magnetron sputtering using Ni64Zr36 and V/Nb targets. The overall thickness was adjusted to 2 μm, and that of the internal V or Nb layer was varied between 0.5 and 1.6 μm. The hydrogen permeability was measured in the range of 373–573 K. All the multi-layered thin films showed negative activation energy for hydrogen permeation indicating an increasing permeability with a decreasing temperature. The calculated negative activation energy, −0.43 to −0.60 kJ/mol for Ni64Zr36/V/Ni64Zr36 thin films and −1.19 to −3.90 kJ/mol for Ni64Zr36/Nb/Ni64Zr36 thin films, and the increased hydrogen permeation at low temperature region is obviously due to the V or Nb mid-layer. The observed hydrogen permeability of the prepared thin films was higher than the reported value of amorphous Ni64Zr36 alloy membrane at 573 K and reached the permeability of Pd at 373 K.