x mol% CeO2-YTaO4 (x = 0, 3, 6, 9, 12) ceramics have been synthesized by the spark plasma sintering (SPS) technique. We focus on the changes in lattice distortion, bonding length, thermal conductivity, thermal expansion, and phase stability of the prepared samples. XRD, Raman, and XPS are used to determine the chemical valence and solid solution mechanism of Ce in the lattice of YTaO4, while its effects on thermal/mechanical properties are elucidated from microstructures. Y3+ is substituted via Ce3+, and all samples maintain a monoclinic phase. The limit thermal conductivity (1.2 W∙m−1∙K−1, 900 °C) is realized in 9 mol% CeO2-YTaO4, and the thermal expansion coefficients are increased to 10.2 × 10−6 K−1 at 1200 °C. Furthermore, the exceptional phase stability and mechanical properties of all samples indicate that they can provide good thermal insulation at high temperatures, and have higher working temperatures than the current YSZ thermal barrier coatings.