Cancer-related fatigue (CRF) is well documented in cancer survivors, but little is known about the personal and societal impact of CRF. This study aimed to examine the impact of CRF in relation to social and vocational functioning and health care utilization in a large sample of post-treatment cancer survivors. We conducted a cross-sectional descriptive study of early stage breast and colorectal cancer survivors (n = 454) who were within 5 years from treatment completion. Social difficulties (SDI-21), work status, absenteeism and presenteeism (WHO-HPQ) and healthcare utilization (HSUQ) were compared in those with (CFR +) and without (CRF -) clinically significant fatigue (FACT-F ≤ 34). A total of 32% met the cut-off criteria for CRF (≤ 34). Participants with CRF + had significantly higher scores on the SDI-21 across all domains and 55% of CRF + vs. 11% in CRF - was above the SDI cut-off (> 10) for significant social difficulties. Participants with CRF + were 2.74 times more likely to be unemployed or on leave (95% CI 1.62, 4.61, p < 0.001). In the subgroup of participants who were currently working (n = 249), those with CRF + reported working on average 27.4 fewer hours in the previous 4 weeks compared to CRF - (p = 0.05), and absolute presenteeism was on average 13% lower in the CRF + group (95% CI 8.0, 18.2, p < 0.001). Finally, individuals with CRF + reported significantly more physician (p < 0.001), other health care professional (p = 0.03) and psychosocial visits (p = 0.002) in the past month. CRF is associated with substantial disruption in social and work role functioning in the early transitional phase of cancer survivorship. Better management of persistent CRF and funding for the implementation of existing guidelines and recommended evidence-based interventions are urgently needed.