Excess anthropogenic phosphorus in watersheds, transported with runoff, can result in aquatic eutrophication, a serious global water quality concern. Watersheds can retain phosphorus, especially in their soils, which can serve as a buffer against the effect of excessive use of phosphorus. However, whether there is a quantifiable threshold at which a watershed exceeds its optimal phosphorus buffering capacity (beyond which riverine loads would dramatically increase) remains unknown. Here we quantified a watershed phosphorus buffering capacity threshold based on accumulation data over 110 years in 23 watersheds of a large North American river basin with globally representative agricultural soils. We found a surprisingly low threshold of just 2.1 t P km−2 (0.03–8.7 t P km−2). Beyond this, further P inputs to watersheds cause a significant acceleration of P loss in runoff. Using a simple exponential decay model, the time estimated to eliminate legacy P via runoff in our watersheds ranges from ~ 100 to over 2,000 years. The rapidity with which the watershed buffering threshold can be surpassed during accumulation, particularly given current anthropogenic phosphorus input rates, versus the long return to baseline suggests that new strategies to reconcile watershed activities and water quality are urgently needed. Watersheds have a low buffering capacity for phosphorus inputs, and their recovery from phosphorus pollution can take over 2,000 years, according to an analysis of phosphorus data from a large North American river.