ABSTRACTModern electric vehicle (EV) batteries need phase change materials (PCM) that are capable of efficient battery cooling. In this work, a composite PCM is prepared by mixing Fe3O4 nanoparticles (1 wt.%) in paraffin, and the effects of these nanoparticles on the enthalpy and melting point of PCM are studied. It is found that the Fe3O4 nanoparticle additives reduce the onset of melting from 61.46°C to 57.03°C. The composite PCM is used for the cooling of a battery module of 6 substitute‐18 650 batteries, and the cooling performance is experimentally and numerically investigated. The hybrid battery thermal management system (BTMS) utilizing composite paraffin demonstrates a significant reduction of 11.2°C in lithium‐ion battery (LIB) temperature compared with natural convection cooling at a heat generation rate of 2W. The numerical results in this study are in good agreement with the experimental temperature values, with a modest mean absolute error of 1.35°C detected between experimentally obtained and simulated battery temperature values. In order to deal with the low thermal conductivity of liquid PCM after PCM melting, a numerical investigation is conducted to study the effect of a graphite fin on the battery temperature. The use of a fin in hybrid BTMS considerably reduces the temperature of LIBs and temperature difference in the module. The numerical simulations capture the behavior of the phase change phenomenon, showing the evolution of liquid PCM under constant heating. This work presents the dynamic melting patterns of PCM along the length of LIB with and without a fin, which is useful for the effective design of BTMS.