Discoidin Domain Receptor 2 (DDR2) is a receptor tyrosine kinase for collagen, stimulating epithelial-mesenchymal transition and stiffness in breast cancer. Here, we investigated levels of DDR2 in breast tumor cells in relation to vascular invasion, TIL subsets, macrophages, molecular tumor subtypes, modes of detection and prognosis. This retrospective, population-based series of invasive breast carcinomas from the Norwegian Screening Program in Vestfold County (Norway), period 2004–2009, included 200 screening patients and 82 cases detected in screening intervals. DDR2 was examined on core needle biopsies using a semi-quantitative, immunohistochemical staining index and dichotomized as low or high DDR2 expression. Counts of macrophages and TIL subsets were dichotomized based on immunohistochemistry using TMA. We also recorded blood or lymphatic vessel invasion (BVI or LVI) as present or absent by immunohistochemistry. High expression of DDR2 in tumor cells showed significant relation with high counts of CD163+ macrophages (p < 0.001) and FOXP3 TILs (p = 0.011), presence of BVI (p = 0.028), high tumor cell proliferation by Ki67 (p = 0.033), ER negativity (p = 0.001), triple-negative cases (p = 0.038), basal-like features (p < 0.001) as well as interval detection (p < 0.001). By multivariate analysis, high DDR2 expression was related to reduced recurrence-free survival (HR, 2.3, p = 0.017), when examined together with histologic grading, lymph node assessment, tumor diameter, BVI, and molecular tumor subtype. This study supports a link between high DDR2 expression, high counts of macrophages by CD163 (tumor associated) and regulatory T cells by FOXP3 together with the presence of BVI, possibly indicating increased tumor motility and intravasation in aggressive breast tumors.