Magnetic tunnel junctions (MTJs) with ultrathin MgO tunnel barriers are at the heart of magnetic random-access memory (MRAM) and exhibit potential for spin caloritronics applications due to the tunnel magneto-Seebeck effect. However, the high programming current in MRAM can cause substantial heating which degrades the endurance and reliability of MTJs. Here, we report the thermal characterization of ultrathin CoFeB/MgO multilayers with total thicknesses of 4.4, 8.8, 22, and 44 nm, and with varying MgO thicknesses (1.0, 1.3, and 1.6 nm). Through time-domain thermoreflectance (TDTR) measurements and thermal modeling, we extract the intrinsic (∼3.6 W m-1 K-1) and effective (∼0.85 W m-1 K-1) thermal conductivities of annealed 1.0 nm thick MgO at room temperature. Our study reveals the thermal properties of ultrathin MgO tunnel barriers, especially the role of thermal boundary resistance, and contributes to a more precise thermal analysis of MTJs to improve the design and reliability of MRAM technologies.