The development and utilization of geothermal energy can effectively alleviate many environmental issues caused by fossil fuel combustion and the global shortage of fossil energy. Cementing materials have an important influence on the parameters of geothermal wells. Through the orthogonal test, SEM, XRD, MIP, and other methods, the effects of the type, particle size, and content of thermal conductive filler on the performance of cementing materials were studied. High thermal conductive cementing material (HTC) for geothermal wells was prepared. The basic physical properties, micromorphology, phase composition, and pore structure of HTC were tested, and the thermally conductive mechanism of HTC was summarized. The results show that graphite has the best effect on improving the thermal conductivity of cement. With the larger particle size and amount of graphite, the thermal conductivity of cement is greater, but the compressive strength is smaller. The formula of HTC has been obtained as follows: 0.44 W/C ratio, 7 wt% graphite, 4 wt% iron powder, 2 wt% quartz sand, in which the particle sizes of graphite are 150 μm and 100 μm which mix in the ratio of 3:1. The thermal conductivity of HTC is 2.003 W/(m·K), which is 69.5% higher than that of high-sulfate-resistant G-grade oil well cement (HGC), and the 48 h compressive strength is 23.38 MPa, which is 3.07% lower than that of HGC. Compared with HGC, the overall structure of HTC is slightly loose, but the degree of hydration is higher. A large number of hydration products such as Ca(OH)2, C-S-H, and AFt are formed, but the pore structure of HTC is slightly poor. The harmful porosity of HTC is 6.46%, which is 3.20% higher than that of HGC. The thermally conductive mechanism of HTC conforms to the theory of heat conduction path, and it is at the beginning of the third stage. The research results can provide theoretical guidance for the preparation and performance research of similar materials, promoting the efficient development and utilization of deep geothermal energy.