The type and thermophysical properties of heat transfer fluids significantly impact the heat extraction rate of Enhanced Geothermal Systems (EGS). Studies based on field-scale stochastic discrete fracture network geometric models have certain limitations and uncertainties. To compare the heat extraction performance of heat transfer fluids H2O, CO2, and N2O in a core-scale (φ50 × 100 mm) rough single fracture in hot dry rock, we first constructed the fracture geometry with rock surface roughness characteristics using a fractal self-affine function, with a joint roughness coefficient (JRC) of 12.38. Then, at temperatures ranging from 37 to 300°C and a pressure of 30 MPa, we established a thermo-hydraulic (TH) coupling model based on the thermophysical parameter equations of the three heat transfer fluids. Finally, we conducted a comparative study of the changes in the outlet mean temperature and heat extraction rate after flowing through the fracture, as well as the temperature distribution on the fracture surface, under conditions of constant inlet flow velocity with varying inlet temperature and constant inlet temperature with varying inlet flow velocity. The results indicate that (1) with the increase in inlet flow velocity and temperature, the outlet mean temperature and heat extraction rate of CO2 and N2O at the fracture outlet are essentially the same and lower than those of H2O. (2) When the inlet flow velocity increases from 0.005 m/s to 0.025 m/s, the thermal convection effect from the rock matrix to the fluid in the fracture predominates, and flow velocity is the main factor affecting the heat transfer process between the fluid and the rock matrix. When the inlet temperature increases from 37°C to 57°C, the thermal conduction effect from the rock matrix to the fluid in the fracture predominates, and temperature is the main factor affecting the heat transfer process between the fluid and the rock matrix.(3)When maintaining constant inlet temperature, increasing inlet flow velocity from 0.005 m/s to 0.025 m/s results in CO2 experiencing a maximum heat extraction rate increase of 118.85 W. In contrast, N2O experiences 117.05 W, and H2O experiences 266.4 W.(4) When the inlet flow rate is constant, the maximum increase in heat extraction rate for CO2 is 15.34 W as the inlet temperature increases from 37 °C to 57 °C; for N2O, it is 13.9 W; and for H2O, it is 45.45 W.