The immune system plays an important role in tumor development and treatment. In this study, we aimed to determine the relationships among the expressions of PD-L1, CD3, CD8, MMR proteins, clinicopathological features, and prognosis of CRC. Immunohistochemistry was used to determine the expression of PD-L1, CD3, and CD8 in 771 patients with CRC. The expression of PD-L1 in TC was related to the right colon, adenocarcinoma, and dMMR, and in IC, it was related to younger CRC patients and the TNM stage. The expression of CD3 and CD8 in tumor-infiltrating lymphocytes was related to lymph node metastasis and the TNM stage. The expression of PD-L1 in TC and IC was correlated with the infiltration of CD3+ and CD8+ lymphocytes. Univariate survival analysis showed that the expression of PD-L1 in TC, IC, and dMMR was related to a better prognosis. Multivariate survival analysis showed that age, TNM stage, and dMMR were independent prognostic factors for CRC. The OS of the chemotherapy was significantly higher than that of the non-chemotherapy in III-IV TNM stage patients; CRC patients with positive PD-L1 expression in TC or IC and dMMR did not benefit from chemotherapy. PD-L1 expression in TC and IC was closely related to the density of CD3 and CD8 infiltration in tumor-infiltrating lymphocytes. The expression of CD3 and CD8 in tumor-infiltrating lymphocytes and the expression of PD-L1 in IC were linked to the TNM stage of CRC patients. PD-L1 expression in TC and IC and MMR status may act as an important biomarker for guiding the postoperative treatment of III-IV TNM stage CRC patients.