To obtain high-performance electromagnetic shielding materials, structure and morphology are two key factors. We here developed an efficient and facial method to prepare high-performance 3D carbon nanofiber mats (CFM)/Fe3O4 hybrid electromagnetic shielding materials. For this purpose, the CFM were chemically modified by mussel-inspired poly-dopamine coating, which were further used as templates for decoration of Fe3O4 nanoparticles via solvothermal route. It was found that the Fe3O4 nano-spheres with diameters of 200–250 nm were uniformly coated on the surface of 3D carbon nanofibers. More importantly, the morphology and structure of resulting 3D carbon nanofiber mats/Fe3O4 hybrids could be easily controlled by altering the experiment parameters, which were examined by FT-IR, XPS, TGA, XRD, SEM, and TEM. The measured magnetic properties showed that saturation magnetism and coercivity increased from 13.4 to 39.7 emu/g and 85.3 to 104.6 Oe, respectively. The lowest reflectivity of resulting hybrid was calculated to be −47 dB at 10.0 GHz (2.5 mm). In addition, the reflectivity of 3D carbon nanofiber mats/Fe3O4 hybrid was less than −25 dB in the range of 7–13 GHz. Moreover, the resulting 3D carbon nanofiber mats/Fe3O4 hybrid exhibited an EMI shielding performance of −62.6 dB in the frequency range of 8.2–12.4 GHz. Therefore, 3D carbon fiber mats/Fe3O4 hybrids can be ideal EMI materials with strong absorption, low density, and wide absorption range.