The purpose of this study was to examine the predictors of quality of life (QOL) of women with breast cancer in Taiwan. A cross-sectional study assessed the quality of life of 394 women with breast cancer. Path models were used to test the direct and indirect effects of depression, anxiety, sleep quality and demographic factors on the four QOL domains. Mean age of the participants was 53.53 ± 7.82 years. The results of the path analysis showed that depression was the key predictor with direct effects on all four QOL domains from -0.248 to -0.580. Sleep quality was found to have direct effects from -0.171 to -0.278 on three QOL domains. Anxiety predicted the physical QOL directly (-0.249). Anxiety also indirectly predicted the physical and psychological QOLs through sleep quality (-0.088 and -0.054), respectively. As for the demographic factors, age had a negative direct effect (-0.125) on the physical QOL domain and economic status directly predicted the environmental QOL domain (0.225). Education had slight indirect effects of 0.108 and 0.090, respectively, on the physical and psychological QOL domains. The empirically validated causal model suggested that a multidimensional approach may be needed to improve the QOL of women with breast cancer.