Heterogeneous microstructure-induced mechanical responses in EH420 shipbuilding steel welded joint by electro-gas welding processed have been systematically studied by scanning electron microscopy, electron backscatter diffraction and mechanical testing. Comparing with the coarse-grained heat-affected zone (CGHAZ), the weld metal presents higher toughness (129.3 J vs. 37.3 J) as it contains a large number of acicular ferrites with high-angle grain boundaries (frequency 79.2%) and special grain boundary ∑3 (frequency 55.3%). Moreover, coarse austenite grains in CGHAZ and slender martensite–austenite constituents between bainite laths may likely facilitate crack propagation. Polygonal ferrites and tempered pearlites formed at the junction of the fine-grained heat-affected zone and the intercritical heat-affected zone induced a softened zone with an average hardness of 185 HV0.5, which is the main reason for the occurrence of tensile fracture.