Previously, it was shown in patients with low rectal cancer that a short-axis (SA) lateral node size of 7 mm or greater on primary magnetic resonance imaging (MRI) resulted in a high lateral local recurrence (LLR) rate after chemoradiotherapy or radiotherapy ([C]RT) with total mesorectal excision (TME) and that this risk was lowered by a lateral lymph node dissection (LLND). The role of restaging MRI after (C)RT with regard to LLR risk and which specific patients might benefit from an LLND is not fully understood. To determine the factors on primary and restaging MRI that are associated with LLR in low rectal cancer after (C)RT and to formulate specific guidelines on which patients might benefit from an LLND. In this retrospective, multicenter, pooled cohort study, patients who underwent surgery for cT3 or cT4 low rectal cancer with a curative intent from 12 centers in 7 countries from January 2009 to December 2013 were included. All patients' MRIs were rereviewed according to a standardized protocol, with specific attention to lateral nodal features. The original cohort included 1216 patients. For this study, patients who underwent (C)RT and had a restaging MRI were selected, leaving 741 for analyses across 10 institutions, including 651 who underwent (C)RT with TME and 90 who underwent (C)RT with TME and LLND. The main purpose was to identify the factors on primary and restaging MRI associated with LLR after (C)RT with TME. Whether high-risk patients might benefit in terms of LLR reduction from an LLND was also studied. Of the 741 included patients, 480 (64.8%) were male, and the mean (SD) age was 60.4 (12.0) years. An SA lateral node size of 7 mm or greater on primary MRI resulted in a 5-year LLR rate of 17.9% after (C)RT with TME. At 3 years, there were no LLRs in 28 patients (29.2%) with lateral nodes that were 4 mm or less on restaging MRI. Nodes that were 7 mm or greater on primary MRI and greater than 4 mm on restaging MRI in the internal iliac compartment resulted in a 5-year LLR rate of 52.3%, significantly higher compared with nodes in the obturator compartment of that size (9.5%; hazard ratio, 5.8; 95% CI, 1.6-21.3; P = .003). Compared with (C)RT with TME alone, treatment with (C)RT with TME and LLND in these unresponsive internal nodes resulted in a significantly lower LLR rate of 8.7% (hazard ratio, 6.2; 95% CI, 1.4-28.5; P = .007). Restaging MRI is important in clinical decision making in lateral nodal disease. In patients with shrinkage of lateral nodes from an SA node size of 7 mm or greater on primary MRI to an SA node size of 4 mm or less on restaging MRI, which occurs in about 30% of cases, LLND can be avoided. However, persistently enlarged nodes in the internal iliac compartment indicate an extremely high risk of LLR, and an LLND lowered LLR in these cases.