Local conditions where people live continue to influence prostate cancer outcomes. By examining local characteristics associated with trends in Black-White differences in prostate cancer specific mortality (PCSM) over time, we aim to identify factors driving county-level PCSM disparities over a 15-year period. We linked county-level data (Area Health Resource File) with clinicodemographic data of men with prostate cancer (Surveillance, Epidemiology, and End Results registry) from 2005 to 2020. Generalized linear mixed models evaluated associations between race and county-level age-standardized PCSM, adjusting for age, year of death, rurality, and county-level education, income, uninsured rates, and densities of urologists, radiologists, primary care providers, and hospital beds. 185,390 patients in 1085 counties were identified, of which 15.8% were non-Hispanic Black. Racial disparities in PCSM narrowed from 2005 to 2020 (25.4 per 100,000 to 19.2 per 100,000 overall; 57.9 per 100,000 to 38 per 100,000 for Non-Hispanic Black patients and 23.4 per 100,000 to 18.3 per 100,000 for Non-Hispanic White patients). For both Non-Hispanic Black and Non-Hispanic White patients, county PCSM changes varied greatly (-65% to + 77% and -61% to + 112%, respectively). From 2016 to 2020, Non-Hispanic Black harbored greater PCSM risk (RR 2.09, 95% CI 2.01-2.18); higher radiation oncologist density was significantly associated with lower mortality risk (RR 0.93, 95% CI 0.89-0.98) while other provider densities were not. Although overall rates improved, specific counties experienced worsening race-based disparities over time. Identifying locations of highest (and lowest) mortality disparities remains critical to development of location-specific solutions to racial disparities in prostate cancer outcomes.