In order to investigate the formation mechanism of Ce5Co19-type super-stacking structure phase, La0.60M0.20Mg0.20Ni3.80 (M = La, Pr, Nd, Gd) compounds are synthesized by powder sintering method. Rietveld refinements of X-ray diffraction patterns find that La0.80Mg0.20Ni3.80 compound has a single Pr5Co19-type structure. The Ce5Co19-type phase appears and increases with the decrease of atomic radius of M, until the La0.60Gd0.20Mg0.20Ni3.80 compound shows a Ce5Co19-type single phase structure. The cycling stability and high rate dischargeability (HRD) of the alloy electrodes both improve with the increase of Ce5Co19-type phase. The capacity retention of La0.60Gd0.20Mg0.20Ni3.80 compound at the 100th cycle is high to 93.6% and the HRD reaches 66.9% at a discharge current density of 1500 mA g−1. Moreover after 50 charge/discharge cycles, the Ce5Co19-type particle retains an intact crystal structure while severe amorphization occurs to Pr5Co19-type particle as shown in graphical abstract. The cohesive energy obtained from the First-principle calculations is analyzed combined with the experimental results. It is found that the La0.60Gd0.20Mg0.20Ni3.80 compound with Ce5Co19-type single phase structure has the highest cohesive energy indicating a more stable structure. This work provides new insights into the superior composition-structure design of LaMgNi system hydrogen storage alloys that may improve the cycling stability.