Depletion of aquifers across the globe is challenging our ability to maintain critically needed agricultural production and provide potable water supplies for millions. In most cases, the only option to decrease the rate of depletion is to reduce the pumping of groundwater. Although implementation of large-scale pumping reductions in the absence of alternative water sources has proven difficult, recent work has shown that locally based, stakeholder-driven initiatives, coupled with regulatory oversight, can be a promising path forward. A critical question is how much must pumping be reduced to have a significant impact on decline rates. Data limitations and modeling uncertainties, however, have frustrated efforts to answer this question with reliable estimates of the needed reductions. We address this situation using a variant of the water-balance equation to identify a key factor, the misestimation of specific yield, that is limiting our ability to assess the impact of proposed pumping reductions. We find that common modeling practices can lead to large overestimates of the required pumping reductions, thereby inadvertently discouraging conservation efforts. We demonstrate the importance of this general finding using data from the High Plains Aquifer in the central United States where common practices have led to overestimates of required pumping reductions by a factor of three to six. We introduce a new metric, the coefficient of variation of net inflow, to help identify such conditions. The reliability of estimates of the impact of pumping reductions can be greatly improved when the constraints imposed by this new metric are combined with a recently proposed method for estimation of specific yield from field data. The ramifications of these findings are far reaching, as defensible estimates of the impact of proposed pumping reductions are an essential element of efforts to chart more sustainable paths for the world’s heavily stressed aquifers.