This study seeks to evaluate lateral pelvic lymph node (LPLN) and perirectal lymph node (PRLN) status on magnetic resonance imaging (MRI) as potential risk factors for lymph node metastasis. The subjects were 394 patients with lower rectal cancer who underwent MRI prior to mesorectal excision (combined with lateral pelvic lymph node dissection in 272 patients) at 6 institutes. No patients received neoadjuvant therapy. Cases were classified as cN(+) and cN(-) based on the short axis of the largest lymph node ≥5 and <5mm, respectively. LPLN and PRLN status and other clinicopathologic factors were analyzed by multivariate logistic regression. The importance of identified risk factors for lymph node metastasis was examined using the area under the curve (AUC). Independent risk factors for right LPLN metastasis included histopathological grade (G3+G4), pPRLN(+), M1, cLPLN(+) [odds ratio (OR) 10.73, 95% confidence interval (CI) 4.59-27.1], and those for left LPLN metastasis were age (<64), histopathological grade (G3+G4), pPRLN(+), and cLPLN(+) (OR 24.53, 95% CI 9.16-77.7). ORs for cLPLN(+) were highest. The AUCs for right and left cLPLN status of 0.7484 (95% CI 0.6672-0.8153) and 0.7904 (95% CI 0.7088-0.8538), respectively, were significantly higher than those for other risk factors. In contrast, the ORs for cPRLN(+) and cPRLN status of 2.46 (95% CI 1.47-4.18) and 0.6396 (95% CI 0.5917-0.6848) were not much higher than for other factors. An LPLN-positive status with a short axis ≥5mm on MRI is an important predictor of LPLN metastasis, but PRLN status is not a strong predictor of PRLN metastasis.