In order to obtain the microwave absorption (MA) materials with light weight, high efficiency and tunable properties, the carbonized mesophase pitch (CMP) with the variation in carbonization temperatures and particle sizes were prepared and characterized. The carbonization temperature mainly controlled the graphitization degree of the CMP to affect their conductive loss. The carbon residues were generated on the CMP surface when the temperature was higher than 700 °C, which contributed significantly to the polarization loss of the CMP. For scale regulation, the segregation between the particles in the paraffin ring caused by the reduction particles of CMP carbonization at 750 °C (750 CMP) resulted in a significant reduction in conductive losses while improving their impedance matching. The 750 CMP over 300 mesh sieved had the strongest MA properties of −53 dB at 3.49 GHz within 5.5 mm. Moreover, the prepared CMPs were multi-layer compounded and optimized by CST microwave studio. The synergistic effect derived from the improved impedance matching and the enhanced interfacial polarization resulted in significant reflection loss in multi-layer CMP. Overall, these findings lead to the systematically regulation of carbon-based materials for MA, showing an attractive application prospect for the preparation of high-performance MA materials.