The results of previous studies investigating whether there is an association between active smoking and risk of death among breast cancer patients have been inconsistent. We investigated the association between active and passive smoking and risk of all-cause and breast cancer-specific death among female breast cancer patients in relation to menopausal and tumor estrogen/progesterone receptor (ER/PR) status. The present study included 848 patients admitted to a single hospital in Japan from 1997 to 2007. Active or passive smoking status was assessed using a self-administered questionnaire. The patients were followed until 31 December 2010. We used a Cox proportional-hazard model to estimate hazard ratios (HR). During a median follow-up period of 6.7 years, 170 all-cause and 132 breast cancer-specific deaths were observed. Among premenopausal patients, current smokers showed a non-significant higher risk of all-cause and breast cancer-specific death. A duration of smoking >21.5 years was positively associated with all-cause (HR = 3.09, 95% confidence interval [CI], 1.17–8.20) and breast cancer-specific death (HR = 3.35, 95% CI: 1.22–9.23, Ptrend = 0.035) among premenopausal patients. In premenopausal patients with ER+ or PR+ tumors, there was some suggestion that a longer duration of smoking was associated with higher risk of all-cause and breast cancer-specific death. Passive smoking demonstrated no significant risk. Our results suggest that a longer duration of active smoking is associated with an increased risk of all-cause and breast cancer-specific death among premenopausal patients, possibly with hormonal receptor-positive tumors. Breast cancer patients should be informed about the importance of smoking cessation.