To ensure a surgical margin of ≥1 cm for the effective treatment of phyllodes tumors of the breast (PTB) a second resection has been recommended, but the outcomes of an extensive series of cases employing the aforementioned criterion cast doubt on this clinical approach. The aim of this study was to identify the local recurrence (LR) risk factors of PTB and determine future optimal surgical treatment according to verified risks. All cases given a diagnosis of PTB, and resected between 1989 and 2008, were retrospectively evaluated. Clinicopathologic data and clinical outcomes were analyzed and stratified according to the risks for LR. All 285 cases were categorized as benign (191, 67.0 %), borderline (61, 21.4 %), or malignant (33, 11.6 %). Median follow-up was 6.7 years and there were 20 LRs during follow-up. All benign PTB recurred as benign PTB lesions. Mitoses (p < 0.001) and tumor size (p = 0.021) were independent prognostic factors for LR in multivariate analysis. Neither margin status (p = 0.758) nor type of surgery (p = 0.922) had any significance for LR. In the risk stratification for LR, PTB ≤5 cm in size with ≥10 mitoses/10 high-power fields (HPFs) had the highest LR rate (55.6 %) compared with all other subgroups (p < 0.001). It is recommended a wide excision and clear margin of 1 cm be ascertained in only small PTB with frequent mitoses, if necessary by means of a second surgery, which could be considered in order to avoid the risk of LR in this distinct and limited group.