This study developed a structural fatigue life prediction model of thermite welded rail joints taking account of the weld-induced residual stress and stress relaxation by external cyclic load. A three-dimensional finite element model including the sequentially coupled thermal–mechanical simulation and cyclic plasticity constitutive simulation was developed to investigate the welding process induced residual stress and the redistribution of the residual stresses due to cyclic bend fatigue loading, respectively. The numerical results of the finite element model were compared with experimental residual stress data to confirm the accuracy of the model developed. The fatigue crack propagation life prediction taking into account the as-welded residual stress and the stress redistribution was performed. The predicted life of weld-toe cracks at web-to-base location will be overestimated by at least 50% if residual stress and relaxation is not considered.