• A simple preparation method to design carbon fiber and metal oxide heterostructures • ZnO@CF heterostructure is beneficial to electromagnetic wave absorption • The effective bandwidth of ZnO@CF heterostructure achieved 4.94 GHz at low filling content (20wt%) and ultra-thin thickness (1.5mm) • The effective bandwidth covered the whole 2-18 GHz band with the increase of thickness from 0.5 to 10 mm With the aim to obtain enhanced absorbing performance at small thickness and low filling, a robust strategy to fabricate zinc oxide (ZnO) modified carbon fiber (CF) structures have been successfully prepared by using low temperature hydrothermal method. Due to the multi-interface polarization caused by the high specific surface area of the complex heterostructures and the improvement of impedance matching, the composites show excellent electromagnetic wave absorption properties. Under the condition of low filling content (20 wt%) and ultra-thin thickness (1.5 mm), the excellent absorption performance of minimal reflection loss of −34.4 dB and an effective absorption bandwidth (RL ≤ −10 dB) of 4.94 GHz is achieved. In addition, the effective absorption bandwidth covers the whole 2-18 GHz band with the increase of thickness from 0.5 to 10 mm. This work provides an innovative method for designing the matching layer of carbon-based absorbing materials, and ZnO@CF heterostructure is expected to become a potential absorbing material.