As the most brittle areas of welded S690QL high-strength steel structures, coarse-grained heat-affected zone (CGHAZ) was crucial for the structural integrity of components. This paper designed an innovative low-temperature impact testing method with a gradient setting of pendulum angle to investigate the micro-mechanism of crack initiation and propagation in the CGHAZ of S690QL welded joints. The results reveal that continuous blocky M−A constituents were distributed along prior austenite grain boundaries (PAGBs), leading to stress concentration and crack initiation. During low-temperature impact processes, cracks propagate in a transgranular mode along {100} or {110} crystal planes. Due to the interaction between compressive stress and the crack tip, regions with high kernel average misorientation (KAM) such as PAGBs can divert the crack. The coarsening of prior austenite grains in the CGHAZ leads to a diminished hindering effect of PAGBs on crack propagation. The findings elucidate the complex relationship between microstructure and impact toughness of CGHAZ, providing insights into crack behavior and mechanisms in CGHAZ.