The association between perioperative changes in the skeletal muscle index (SMI) and colorectal cancer (CRC) outcomes remains unclear. We aim to explore perioperative change patterns of SMI and evaluate their effects on long-term outcomes in CRC patients. This retrospective cohort study included Stage I-III CRC patients who underwent curative resection between 2012 and 2019. SMI at the third lumbar vertebra level was calculated using computed tomography scans. Optimal cut-off values for SMI were defined separately for males and females and classified as high or low preoperatively and at 3, 6, 9 and 12 months postoperatively. SMI status was further categorized into different perioperative SMI change patterns: highpre-highpost, highpre-lowpost, lowpre-highpost and lowpre-lowpost. The association with recurrence-free survival (RFS) and overall survival (OS) was examined using Cox proportional hazards models. A total of 2222 patients (median [interquartile range] age, 60.00 [51.00-68.00] years; 1302 (58.60%) men; 222 (9.99%) with preoperative low SMI) were evaluated. During a median follow-up of 60 months, 375 patients (16.88%) died, and 617 patients (27.77%) experienced a recurrence. Multivariate Cox model analysis showed that, compared to patients with highpre-highpost, those with highpre-lowpost (HR = 3.32, 95% CI: 1.60-6.51; HR = 2.54, 95% CI: 1.03-6.26; HR = 2.93, 95% CI: 1.19-7.19, all p < 0.05) had significantly worse RFS and OS (HR = 4.07, 95% CI: 1.55-10.69; HR = 4.78, 95% CI: 1.40-16.29; HR = 9.69, 95% CI: 2.53-37.05, all p < 0.05), at postoperative 6, 9 and 12 months, respectively. Patients with lowpre-lowpost were an independent prognostic factor for worse OS at postoperative 12 months (HR = 3.20, 95% CI: 1.06-9.71, p = 0.040). Patients with lowpre-highpost had similar risk of RFS compared to those with highpre-highpost at postoperative 3, 6 and 12 months (HR = 1.49, 95% CI: 0.75-2.98; HR = 1.05, 95% CI: 0.45-2.43; HR = 1.36, 95% CI: 0.31-6.06, all p > 0.05) and similar risk of OS at postoperative 3, 6, 9 and 12 months (all p > 0.05). Patients with a high preoperative SMI that decline postoperatively have poor RFS and OS. Consistently low SMI also correlates with worse OS. Patients with low SMI but increased after resection are not an indicator of better prognosis. Routine measurement of postoperative, rather than preoperative, SMI is warranted. Patients with low SMI are at an increased risk for recurrence and death, especially within the first year after surgery.