Pipe to elbow welds are usually identified as the weakest parts in the pipeline system of ultra-supercritical boilers due to the structural induced stress arising from internal steam pressure, and the constraint of supports and hangers. The finite element (FE) method has been applied to investigate the effect of structural induced stress on creep evolution in pipe to elbow welds. The results show that compressive axial structural induced stress can significantly increase the creep strain near the pipe’s outer surface. In contrast, the creep strain near the pipe’s inner surface is clearly accelerated by tensile axial structural induced stress. Compared with free deformation conditions in the pipe ends, when subject to a compressive axial structural induced stress under −30 MPa, the equivalent creep strain in the fine-grained heat affected zone (FGHAZ) at the 12:00 position on the outer surface increases by about 13.7 times. In the case of a 30 MPa tensile axial structural induced stress, the equivalent creep strain increases by about 83.3% in the FGHAZ at the 12:00 position on the inner surface. The maximum creep strain of the pipe to elbow weld in the ultra-supercritical boiler after creep for 5000 h is 1.9% and located at the 10:30 position in the FGHAZ on the pipe’s outer surface, which makes it the weakest part of the welded joint. The location of a crack in a pipe to elbow weld after running for 20,000 h is in agreement with the simulation results.