FeSiCr soft magnetic composites (SMCs) were fabricated by the sol-gel method, and an Al2O3/resin composite layer was employed as the insulation coating. By the decomposition of boehmite (AlOOH) gel into Al2O3 in the temperature range of 606–707 °C, a uniform Al2O3 layer could be formed on the FeSiCr powder surface. The Al2O3 insulation coating not only effectively reduced the core loss, increased the resistivity, and improved the quality factor, but it also increased the thermal conductivity of SMCs. The best overall properties with saturation magnetization Ms = 188 emu/g, effective permeability μe = 39, resistivity ρ = 8.28 × 105 Ω·cm, quality factor Q = 94 at 1 MHz, and core loss = 1173 mW/cm3 at 200 kHz and 50 mT were obtained when the SMC was prepared with powders coated by 0.5 wt.% Al2O3 and resin. The optimized SMC exhibited the lowest core loss with 27% reduction compared to the resin only-insulated sample and 71% reduction compared to the sample without insulation treatment. Importantly, the thermal conductivity of the SMCs is 5.3 W/m·K at room temperature, which is higher than that of the samples prepared by phosphating and SiO2 coating owing to the presence of a high thermal conductive Al2O3 layer. The high thermal conductivity is beneficial to enhancing the high temperature performance, lifetime, and reliability of SMCs. This work is expected to be a valuable reference for the design and fabrication of SMCs to be applied in high-temperature and high-frequency environments.