As a critical parameter, thermal conductivity directly determines the heat transfer and temperature variation within rocks, which can lead to mechanical damage and chemical corrosion. Consequently, understanding the thermal conductivity of stone heritage is vital for assessing their deterioration mechanisms and developing effective conservation strategies. This study obtained sandstone samples from the Yungang Grottoes and subjected them to freeze–thaw cycle experiments to generate weathered sandstone samples. Subsequently, the thermal conductivity of these samples was measured under both dry and water-saturated state using the transient plane source method. To analyze the relationship between air–water saturation, porosity, and thermal conductivity, a saturation influence coefficient was introduced. Thereafter, the effectiveness and applicability of 13 commonly used thermal conductivity mixing law prediction models were evaluated based on experimental data. The results suggested that the influence of water saturation on the thermal conductivity of rocks varies with porosity, and water saturation significantly enhances the thermal conductivity of weathered sandstone. Among the 13 common models, the Geometric mean model was found to be more accurate than other models, with superior performance in both dry (MAE, RMSE, MAPE are 0.148, 0.214, 5.59% respectively) and water-saturated (MAE, RMSE, MAPE are 0.244, 0.170, 8.4% respectively) state. The Albert model demonstrates a good fit in the dry state, whereas the Walsh model (with maximum effect), Ribaud model, and Huang model also exhibit good fitting efficacy in the water-saturated state. This study provides a solid foundation for better predicting the thermal conductivity of weathered stone heritage and developing effective preventive conservation strategies.