The influence of Ti–Mg and Ti on the impact toughness of the heat affected zone in steel has been studied comparatively. The changes of microstructure and grain size in the heat affected zone are analyzed. It has been found that the content of acicular ferrite in the steel treated by Ti–Mg is more than that in the steel treated by Ti. The size of the primary austenite grain in Ti–Mg treated steel is smaller than that in the steel treated by Ti. During the welding thermal cycle, the second phase particles in steel can be re‐dissolved and coarsened. After welding, the second phase particles in Ti–Mg treated steel are smaller than those in Ti treated steel, and the number of the second phase particles in Ti–Mg treated steel is more than that in Ti treated steel. The pinning effect of the second phase particles in Ti–Mg treated steel is obvious stronger than that in Ti treated steel. The impact toughness of the heat affected zone has a significant improvement after the addition of Ti–Mg.