AbstractThe low thermal conductivity (κ) has a significant impact on the application of thermal barrier coatings (TBCs). Rare‐earth tantalates (RETaO4) are one kind of the most promising TBC materials with low thermal conductivity. However, the underlying mechanism of low κ in RETaO4 has remained a mystery. In this work, the thermal transport properties of monoclinic (m)‐RETaO4 (RE = Y, Eu, Gd, Dy, Er) compared with ZrO2 are conducted to reveal the mechanism of low lattice thermal conductivity in the former compounds using highly accurate phonon Boltzmann transport equation combined with first‐principles calculations. The predicted κ is in good agreement with experimental data, which proves that this work is convincing. The result shows that ErTaO4 has the lowest κ (1.37 W m−1 K−1 at 1600 K), which is much lower than ZrO2 (2.49 W m−1 K−1 at 1600 K). It is found that the strong anharmonicity and large scattering rate in m‐RETaO4 are mainly derived from strong ionic bonding in the crystal structure, and strong anti‐crossing property of acoustic‐optical phonon branches in phonon dispersion. Both mechanisms can effectively reduce the phonon group velocity and increase the phonon scattering rates of m‐RETaO4, leading to lower κ than ZrO2. Fortunately, two descriptors, including distortion degree and stretching force constant, are suggested to be used to quickly screen the doping or multicomponent RETaO4 with relatively lower κ, which could also be extended to other potential TBCs systems, that is, rare‐earth silicate, rare‐earth cerate, and so on.