The role of macrophages in colorectal cancer tumorogenesis is complex because they can both prevent and promote tumor development. We investigated CD68-positive cell infiltration in tumor tissue and its correlations with proteins of TGF-β1 signaling pathway and survival of the patients after surgical therapy. A non-selected panel of 210 primary tumors of colorectal origin was investigated immunohistochemically with antibodies against CD68, TGF-β1, TGFβRII and Smad4. Lower CD68 infiltration in tumor stroma was associated with expression of TGF-β1 (p = 0.002) and SMAD4 (p = 0.090) in tumor cell cytoplasm and with TGFβRII expression (p = 0.017) in tumor cells membranes. The absence of SMAD4 immune deposits in tumor cell nuclei was more often seen in biopsies with low number of CD68 in the invasive front (p = 0.044). The low number of CD68-positive cells was significantly associated with several adverse clinical and histological tumor characteristics as the presence of metastases in local lymph nodes (p = 0.047), distant metastases (p = 0.0003), advanced tumor stage (p = 0.006), tumor cell invasion of blood, lymph vessels or perineural invasion (p = 0.004), higher histological types (p = 0.0002) and lower grade of inflammatory infiltration in the invasive front (p = 0.002). Moreover, the low grade of CD68 appeared to be significant unfavorable factors of prognosis of the patients with colorectal cancer. The results of our study confirm the prognostic significance of low level of tumor-associated macrophage infiltration in colorectal cancer as unfavorable marker for survival of the patients.