Welding residual stresses impose significant influences on the fatigue behavior and structural integrity of welded connections in various engineering applications. Understanding the effects of these residual stresses on fatigue life is crucial for ensuring the reliability and safety of welded structures. This paper proposes a damage-incorporated thermal–mechanical simulation method to examine the effect of welding residual stresses on the fatigue life prediction of welded plates and circular hollow section (CHS) X-joints. Compared to the experimental X-ray diffraction measurements and the fatigue tests, the predicted residual stresses and fatigue life demonstrate good agreement with experimental results for both high-cycle and low-cycle fatigue. The location of fatigue cracks resembles closely the estimation from a uniform fatigue damage indicator. The fatigue life prediction without incorporating the welding residual stresses does not affect significantly the life estimation for low-cycle fatigue but can cause substantial over-predictions for high-cycle fatigue.