Care coordination is a strategy to reduce healthcare navigation challenges for cancer patients. The objectives of this study were to assess the association between having a cancer care coordinator (CCC) and long-term health-related quality of life (HRQoL), and to evaluate whether this association differed by level of health literacy. A population-based sample of survivors diagnosed with breast, prostate, or colorectal cancer in 2015 from the Iowa Cancer Registry participated in an online survey conducted in 2017-2018 (N = 368). Chi-squared tests and logistic regression were used to model the association between patient characteristics and having a cancer care coordinator. Linear regression was used to model the association between patient perception of having a cancer care coordinator and post-treatment physical or mental HRQoL by differing levels of health literacy while controlling for sociodemographic and clinical factors. Most survivors (81%) reported having one healthcare professional who coordinated their cancer care. Overall, patient perception of having a coordinator was not significantly associated with physical HRQoL (p = 0.118). However, participants with low health literacy (21%) who had a coordinator had significantly higher physical HRQoL scores compared to those who did not (adjusted mean difference 5.2, p = 0.010), while not so for medium (29%) or high (51%) health literacy (p = 0.227, and p = 0.850, respectively; test for interaction p = 0.001). Mental HRQoL was not associated with having a coordinator in our analyses. Findings suggest that care coordinators improved post-treatment physical HRQoL, particularly for participants with low health literacy. Care coordinators may be beneficial to the most vulnerable patients struggling to navigate the complex healthcare system during cancer treatment. Future research should focus on the mechanisms by which care coordination may affect post-treatment HRQoL.