Microstructures dominate mechanical properties of steel materials. Monitoring and controlling microstructure variations are important for high-quality steel productions. In this study, the multi-frequency electromagnetic technology is used to effectively characterize changes of steel microstructures. Steel samples with individual micro- components are obtained using different heat treatments and are measured using a self-designed multi-frequency electromagnetic system. Micro- and Macro- FE models are developed to discuss electromagnetic responses on steel microstructures with different phases, grain size and grain boundaries. The relative permeability of different steel microstructures are calculated and discussed. Results show that the low frequency inductance will increase with the increase of ferrite grain size and ferrite fraction. The relative permeability of single ferrite grain boundary is determined to be 165 and the orientation of grain boundary can influence the relative permeability of steel microstructures.