Chronic pain is a major complication following breast surgery including breast reconstruction. We previously examined prospective patient-specific and medical/surgical factors that predict chronic pain a year after breast surgery in the Japanese population. Five-year survivorship is essential for breast cancer patients. This report is a 4-year follow-up study following the previous research. A follow-up observation study was performed 5years after breast operations. The subjects were patients who underwent breast surgery, including tissue expander/implant (TE/implant), DIEP procedures and mastectomy only. Pain at 5years was assessed using the Japanese Version of the Short-Form McGill Pain Questionnaire (SF-MPQ-JV). A multiple linear regression model was used to examine the relationships of clinical factors with chronic pain. Questionnaires were completed by 132 subjects. No factor related to chronic pain was significantly related to the MPQ pain ratings. Among patient characteristics, a psychotic or neurological medical history was related to significantly lower visual analog scale (p = 0.02) and present pain index (p = 0.04) scores. A history of chemotherapy and/or hormone therapy was significantly associated with the frequency of use of pain medication postoperatively (p = 0.05) and effect on the social life of the patients (p = 0.02). A psychotic or neurological history and a history of chemotherapy and/or hormone therapy were identified as risk factors for chronic pain after breast surgery, but the type of operation was not associated with chronic pain.