In this study, a butt-welded joint between P92 ferritic steel and SUS304 austenitic stainless steel was fabricated with a nickel-based filler material. Based on Sysweld software, three-dimensional thermal-metallurgical-mechanical coupled finite element analyses were performed to investigate the hardness and welding residual stress in the welded joint. In the material behavior model, various factors influencing the formation and distribution of hardness and welding residual stress were carefully considered, including the austenitic transformation, martensitic transformation, and the tempering of martensite in P92 steel; the isotropic strain hardening, annealing effect, and melting effect of all materials; and the differential thermal expansion between dissimilar metals. The neutron diffraction method and hole-drilling strain gauge method were applied to measure the profiles of residual stress of the welded joint, respectively. Hardness mapping was performed on a weld cross-section specimen. The predicted results of welding residual stress and hardness are in good agreement with the measurements, respectively. The peak residual stresses in longitudinal and transverse directions are above 600 MPa in P92 steel. There are high tensile residual stress and significant cold work near the weld root in SUS304 stainless steel. The solid-state phase transformation induces compressive longitudinal residual stress in the heat-affected zone, high tensile transverse residual stress near the weld toe in P92 steel, and a significant stress gradient. The tempering effect can reduce the hardness and tensile residual stress in the heat-affected zone of P92 steel. The formation mechanism of welding-induced residual stress was illustrated based on the Satoh test. The effects of peak temperature, internal restraint, and differential thermal expansion are found to be decisive for the formation of residual stress. Large tensile residual stress can be generated with internal restraints, which promote the stress increase and lead to material yielding. A restraint factor was proposed for the quantitative assessment of stress evolution.