The effect of cooling rate and Nb addition on the phase transformation, microstructures, and mechanical properties of thermomechanically controlled hot-rolled low-carbon CrNiMnMoB ultrahigh-strength steel were studied. The aspects impeding Nb's benefits on hardenability, phase transformation, and mechanical properties were investigated. The hot rolling process was simulated, and a continuous cooling transformation diagram was constructed for the studied steels. Microstructures were analysed using laser scanning confocal microscopy and field emission scanning electron microscopy, with the latter also used for in-depth investigations of the non-metallic inclusions. Transmission electron microscopy was employed to investigate the precipitate characteristics. The concentrations of Nb in the solution and precipitates were quantified for the martensitic and granular bainitic structures. The effect of Nb in solution on phase transformation and strengthening is diminished, and the hardness is reduced by Nb precipitation at an early processing stage. Decreasing the cooling rate promotes the formation of granular bainitic structures rather than martensite. After being subjected to hot rolling followed by water quenching and air cooling, the microstructures are autotempered martensite and fully granular bainite, respectively. Various non-metallic inclusions, including Al2O3, MnS, and Al2O3–MnS, MnS·BN, Al2O3·BN and MnS·Al2O3·BN were observed. The number density and area fraction of non-metallic inclusions in the Nb-containing steel and the air-cooled samples are higher than those observed in the Nb-free and water-quenched samples. Adding 0.04 wt% of Nb slightly increased the strength, but the toughness properties dropped, mainly due to the higher number density and area fraction of non-metallic inclusions and primary large precipitates.