Based on the decorrelation calculation of diffusion ultrasound in time-frequency domain, this paper discusses the repeatability and potential significance of Disturbance Sensitive Zone (DSZ) in time-frequency domain. The experimental study of Barely Visible Impact Damage (BVID) on Carbon Fiber Reinforced Polymer (CFRP) is carried out. The decorrelation coefficients of time, frequency, and time-frequency domains and DSZ are calculated and compared. It has been observed that the sensitivity of the scattered wave disturbance caused by impact damage is non-uniformly distributed in both the time and frequency domains. This is evident from the non-uniform distribution of the decorrelation coefficient in time-domain and frequency-domain decorrelation calculations. Further, the decorrelation calculation in the time-frequency domain can show the distribution of the sensitivity of the scattered wave disturbance in the time domain and frequency domain. The decorrelation coefficients in time, frequency, and time-frequency domains increase monotonically with the number of impacts. In addition, in the time-frequency domain decorrelation calculation results, stable and repetitive DSZ are observed, which means that the specific frequency component of the scattered wave is extremely sensitive to the damage evolution of the impact region at a specific time. Finally, the DSZ obtained from the first 15 impacts is used to improve the decorrelation calculation in the 16-th to 20-th impact. The results show that the increment rate of the improved decorrelation coefficient is 10.22%. This study reveals that the diffusion ultrasonic decorrelation calculation improved by DSZ makes it feasible to evaluate early-stage damage caused by BVID.