Reliable electromagnetic interference (EMI) shielding is essential to secure communications, protect components, and ensure efficient 5G network operation as these technologies expand. In this investigation, we propose a composite material that demonstrates remarkable EMI shielding efficacy, particularly within the Ka-band frequency range (26–40 GHz). This material is practical for industrial fabrication because it can be readily produced by integrating carbonyl iron powder (CIP) and carbon fabric (CF) into a polydimethylsiloxane matrix. It achieves a high shielding effectiveness of at least 120 dB within the Ka-band range, which can be further increased to an ultra-high level of 160 dB by adjusting the CF content. Electromagnetic analyses reveal that this enhanced shielding efficiency is due to the synergistic interplay between multiple reflections of incident EM waves within the CF structure and the excellent EM wave absorption properties of CIP. This composite material not only enhances EM shielding efficiency but also provides enhanced mechanical strength and thermal conductivity, rendering it a practical choice for EM shielding applications in the 5G Ka-band frequency range.