In the construction industry, falls, slips, and trips (FST) account for 42.3% of all accidents. The primary cause of FST incidents is directly related to the deterioration of workers' body stability. To prevent FST-related accidents, it is crucial to understand the interaction between physical fatigue and body stability in construction workers. Therefore, this study investigates the impact of fatigue on body stability in various construction site environments using Dynamic Time Warping (DTW) analysis. We conducted experiments reflecting six different fatigue levels and four environmental conditions. The analysis process involves comparing changes in DTW values derived from acceleration data obtained through wearable sensors across varying fatigue levels and construction environments. The results reveal the following changes in DTW values across different environments and fatigue levels: for non-obstacle, obstacle, water, and oil conditions, DTW values tend to increase as fatigue levels rise. In our experiments, we observed a significant decrease in body stability against external environments starting from fatigue Levels 3 or 4 (30% and 40% of the maximum failure point). In the non-obstacle condition, the DTW values were 9.4 at Level 0, 12.8 at Level 3, and 23.1 at Level 5. In contrast, for the oil condition, which exhibited the highest DTW values, the values were 10.5 at Level 0, 19.1 at Level 3, and 34.5 at Level 5. These experimental results confirm that the body stability of construction workers is influenced by both fatigue levels and external environmental conditions. Further analysis of recovery time, defined as the time it takes for body stability to return to its original level, revealed an increasing trend in recovery time as fatigue levels increased. This study quantitatively demonstrates through wearable sensor data that, as fatigue levels increase, workers experience decreased body stability and longer recovery times. The findings of this study can inform individual worker fatigue management in the future.