Impact toughness of an ultrafine-grained (UFG) interstitial-free (IF) steel produced by equal-channel angular extrusion/pressing (ECAE/P) at room temperature was investigated using Charpy impact tests. The UFG IF steel shows an improved combination of strength and impact toughness compared with the corresponding coarse-grained (CG) one. The CG IF steel samples underwent a transition in fracture toughness values with decreasing temperature because of a sudden change in fracture mode from microvoid coalescence (ductile) to cleavage (brittle) fracture. Grain refinement down to the submicron (≈320 nm) levels increased the impact energies in the upper shelf and lower shelf regions, and it considerably decreased the ductile-to-brittle transition temperature (DBTT) from 233 K (−40 °C) for the CG steel to approximately 183 K (−90 °C) for the UFG steel. Also, the sudden drop in DBTT with a small transition range for the CG sample changed to a more gradual decrease in energy for the UFG sample. The improvement in toughness after UFG formation was attributed to the combined effects of grain refinement and delamination and/or separation in the heavily deformed microstructure. Although an obvious change from the ductile fracture by dimples to the brittle fracture by cleavage was recognized at 233 K (−40 °C) for the CG steel, no fully brittle fracture occurred even at 103 K (−170 °C) in the UFG steel.