Diabetes mellitus (DM) can worsen the prognosis or survival in prostate cancer (PC) patients. We investigated whether glycemic control impacts mortality in PC patients with existing diabetes. All PC patients with or without preexisting DM were enrolled from 2006 to 2017. Mean hemoglobin A1c (HbA1c) values (<7%, 7%-9%, ≥9%) were used to represent glycemic control. Major outcomes included all-cause, PC-specific, and non-PC mortalities. Statistical analyses were performed using Cox regression models with adjusted mean HbA1c and other related confounders. A total of 831 PC patients were enrolled (non-DM group, n = 690; DM group with a record of mean HbA1c values, n = 141). Results showed that the DM group with mean HbA1c level ≥ 9% (n = 14) had significantly increased risk for all-cause and non-PC mortality (hazard ratio [HR], 3.09; 95% CIs, 1.15-8.32; p=0.025 and HR, 5.49; 95% CIs, 1.66-18.16; p = 0.005, respectively), but not for PC-specific mortality (HR, 1.03; 95% CIs, 0.13-8.44; p = 0.975), compared with the non-DM group. Our findings indicate that PC patients with DM who had a mean HbA1c level ≥ 9% had higher risks of all-cause and non-PC mortality compared with non-DM subjects. Further large and long-term studies are needed to verify the effect of glycemic control in PC patients with DM.