The specific nanostructure of the absorber is a key factor in improving the electromagnetic absorption performance. Herein, NiFe layered double hydroxide composited with carbonyl iron powder (LDH/CIP) was fabricated by a one-step hydrothermal synthesis process for the first time. By varying the morphology of the CIP template (spherical or flaky), the nanostructure of in situ grown LDH could be modulated, to optimize the electromagnetic wave absorption performance of LDH. The resultant NiFe/CIP synergistic composite has proved its excellent electromagnetic (EM) wave absorption performance with the minimum reflection loss (RLmin) of −96.5 dB and an effective absorption bandwidth (EAB) up to 7.6 GHz at the thickness of 8 mm. At a smaller thickness of 3 mm, the EAB value remains 7.2 GHz with the RLmin of −16.4 dB. Such an outstanding absorption capacity is primarily attributed to the magnetic loss of NiFe LDH, which is more significantly enhanced using CIP as the substrate. Moreover, the formation of nano-flower-like LDHs, which possess rich heterogeneous structural interfaces and specific surface area to induce the polarization relaxation of the absorber. Therefore, the established NiFe/CIP composite brings insights into the design of high-performance EM wave absorbers.