Core Ideas We validated a multiphase flow model for simulating heat transfer in an SBTES system. Convective/conductive heat transfer ratio was higher in the middle of heat storage domain. About 10% of the total heat transfer was in the form of latent heat. An asymmetric heat injection scheme can be useful to increase latent heat transfer. A constant effective thermal conductivity in the model can introduce significant errors. In this study, we numerically and experimentally evaluated heat transfer in soils under unsaturated conditions in the context of simulating a laboratory‐scale, three‐dimensional soil‐borehole thermal energy storage (SBTES) system. Most previous studies assumed that soil thermal and hydraulic properties are constant and that heat transfer in soil occurs only in the form of conduction, neglecting convective and latent heat transfer. In addition, there is a lack of data from controlled experiments to validate multiphase numerical models that can be used to better study SBTES systems installed in the vadose zone. The goal of this study was to evaluate the significance and impact of variable soil thermal and hydraulic properties, as well as different heat transfer mechanisms, in unsaturated soils. Four laboratory experiments were performed using a three‐dimensional laboratory‐scale SBTES system, incorporating sensors to collect soil temperature and moisture data at high spatial and temporal resolutions. Experimental data were then used to validate a numerical model that solves for water and vapor flow and considers nonequilibrium phase change. Results revealed that for the test conditions studied, convective heat transfer was higher than conductive heat transfer in the middle of the borehole array. Moreover, for the experiments on unsaturated sand, about 10% of the total heat transfer was in the form of latent heat. Simulation results demonstrated the importance of including both convection and latent heat in SBTES system modeling. Results also revealed a need for using saturation‐dependent effective thermal conductivity in modeling SBTES systems in unsaturated soils rather than using constant values such as those obtained from system thermal response tests.