Nowadays, lithium-ion batteries are widely used in electric vehicles as the power source and its safety attracts increasing attention. Particularly, the thermal runaway is a highest risk needed to be solved. In order to effectively inhibit thermal runaway propagation, an efficient and energy-saving battery thermal management system is proposed in this study, which integrates phase change cooling, nanofluid cooling and heat insulation materials. Firstly, the system model is established by integrating electrochemical model, thermal model and fluid model, and the validity of the model is analyzed. Then the cooling performance of the system under two working conditions is discussed. In the normal heat dissipation condition, Scheme 6 can effectively inhibit thermal runaway propagation and reduce the maximum battery temperature from 1013.50 K to 328.34 K. In the extreme condition, Scheme 6 can reduce the maximum battery temperature from 980.51 K to 380.34 K, and the heat in the system can be taken away in time. In addition, the effects of the volume fraction of nanoparticles and the flow rate of nanofluids on the cooling performance are studied. Finally, for purpose of further improving the cooling performance, uniform-precision rotatable central composite design is used to establish the regression equation and determine the best combination factors. Comparing with Scheme 6, the maximum battery temperature in the improved scheme is reduced by 23%, and the economic index is reduced by 22%.