In order to further optimize welding process of Nb–Ti–Mo microalloyed steel, welding thermal cycles on coarse-grained heat-affected zone (CGHAZ) of welded joints were simulated using Gleeble 1500. The microstructure and low-temperature impact fracture were investigated using a scanning electron microscope and a pendulum impact machine, respectively. Moreover, the relationship between cooling time t8/5 and the microstructure of CGHAZ was discussed, and the effect of microstructure on impact toughness was also studied. As cooling time increased, martensite fraction decreased from 97.8% (3 s) to 3.0% (60 s). The fraction of martensite/austenite (M/A) constituent increased from 2.2% (3 s) to 39.0% (60 s), its shape changed from granular to strip, and the maximum length increased from 2.4 μm (3 s) to 7.0 μm (60 s). As cooling time increased, the prior austenite grain size increased from 34.0 μm (3 s) to 49.0 μm (60 s), the impact absorption energy reduced from 101.8 J (5 s) to 7.2 J (60 s), and the fracture mechanism changed from quasi-cleavage fracture to cleavage fracture. The decreased toughness of CGHAZ was due to the reduction of lath martensite-content, coarsening of original austenite grain, and increase and coarsening of M/A constituent. The heat input was controlled under 7 kJ cm−1 during actual welding for these steels.