In this study, a new strategy was utilized to fabricate Mg-based hydrogen storage fibers with uniform elemental distribution and diameters of 30–50 μm. The in-situ formation of nano-precipitates in Mg85Ni14.8Ag0.2 metallic glass fibers was visually observed at varied annealing temperatures. It indicates the metastable Mg6Ni phase is formed from an amorphous microstructure initially. Then, the needle-like Mg2Ni nanoparticles precipitate at 140–160 °C and subsequently grow into spherical shapes. This results in the formation of the fiber structure with well-distributed nano-Mg2Ni particles. The alloy fibers can absorb H2 at only 40 °C, and the activation energy for hydrogenation is lowed to be 46.9 kJ/mol. The onset desorption temperature of hydride is reduced to 220 °C due to the improved the catalytic efficiency by uniformly distributed Mg2Ni nanoparticles. The abundant interfaces, as well as the shortened distance by micropores, serve as paths for the fast diffusion of H atoms.