A single nitrogen-vacancy (NV) center in a diamond can be used as a nanoscale sensor for magnetic field, electric field or nuclear spins. Due to its low photon detection efficiency, such sensing processes often take a long time, suffering from an electron spin resonance (ESR) frequency fluctuation induced by the time-varying thermal perturbations noise. Thus, suppressing the thermal noise is the fundamental way to enhance single-sensor performance, which is typically achieved by utilizing a thermal control protocol with a complicated and highly costly apparatus if a millikelvin-level stabilization is required. Here, we analyze the real-time thermal drift and utilize an active way to alternately track the single-spin ESR frequency drift in the experiment. Using this method, we achieve a temperature stabilization effect equivalent to sub-millikelvin (0.8 mK) level with no extra environmental thermal control, and the spin-state readout contrast is significantly improved in long-lasting experiments. This method holds broad applicability for NV-based single-spin experiments and harbors the potential for prospective expansion into diverse nanoscale quantum sensing domains.