The construction of transition metal nanocatalysts has become an attractive way to improve the hydrogen storage performance of MgH2. However, it is still a great challenge to obtain multielement transition metal nanocatalysts, improve the hydrogen storage capacity of MgH2 under mild conditions (<573 K), and reduce the apparent activation energy of hydrogen absorption and desorption. In this work, a flower-like CoNi-MOF was successfully synthesized by sacrificing templates. After further pyrolysis, we obtained hierarchical structure flower-like MOF derivatives assembled by CoNi@C nanoparticles with an average particle size of 15.1 nm. Then, MgH2-x wt % CoNi@C nanocomposites are fabricated through the ball milling process. Due to the unique hierarchical flower-like structure of MOF derivatives, the inhibition of an amorphous carbon shell on CoNi alloy growth and agglomeration, and the synergistic catalysis of Mg2Co and Mg2Ni, the MgH2-10 wt % CoNi@C nanocomposite exhibits excellent hydrogen storage capacity under mild conditions and low apparent activation energy: At 473 K, the nanocomposite can quickly absorb 5.0 wt % H2 in 5 min, and even at 373 K, it can still absorb 4.2 wt % H2 in 60 min. At 573 and 548 K, it can release 4.8 and 3.1 wt % H2, respectively. The apparent activation energies for hydrogen absorption and desorption decrease remarkably to 24.9 and 67.3 kJ/mol, respectively, which are significantly better than those for MgH2-10 wt % Co@C (44.7 and 79.8 kJ/mol) and MgH2-10 wt % Ni@C (39.0 and 78.9 kJ/mol) nanocomposites. The first-principles calculation indicated that the hydrogen adsorption energy of the Mg–CoNi model is as low as −5.87 eV. This work provides a new strategy for the design of highly efficient multielement transition metal hydrogen storage material catalysts with a hierarchical structure.