Newly diagnosed breast cancer patients face substantial stress and uncertainty that may undermine their quality of life (QoL). The purpose of the present study was to examine the associations between health-related fitness (HRF) and QoL in newly diagnosed breast cancer patients from the Alberta Moving Beyond Breast Cancer Study. Newly diagnosed breast cancer patients with early-stage disease (n = 1458) were recruited between 2012 and 2019 in Edmonton and Calgary, Canada to complete baseline HRF and QoL assessments within 90days of diagnosis. HRF assessments included cardiorespiratory fitness (VO2peak treadmill test), muscular fitness (upper and lower body strength and endurance tests), and body composition (dual x-ray absorptiometry). QoL was assessed by the Medical Outcomes Study Short Form 36 (SF-36) version 2. We used logistic regression analyses to examine the associations between quartiles of HRF and poor/fair QoL (bottom 20%) after adjusting for key covariates. In multivariable analysis, the least fit groups compared to the most fit groups for relative upper body strength (OR = 3.19; 95% CI = 1.98-5.14), lean mass percentage (OR = 2.31; 95% CI = 1.37-3.89), and relative VO2peak (OR = 2.08; 95% CI = 1.21-3.57) were independently at a significantly higher risk of poor/fair physical QoL. No meaningful associations were found for mental QoL. The three main components of HRF (muscular fitness, cardiorespiratory fitness, and body composition) were independently associated with physical QoL in newly diagnosed breast cancer patients. Exercise interventions designed to improve these components of HRF may optimize physical QoL and help newly diagnosed breast cancer patients better prepare for treatments and recovery.