Effective policies for adaptation to climate change require understanding how impacts are related to exposures and vulnerability, the dimensions of the climate system that will change most and where human impacts will be most draconian, and the institutions best suited to respond. Here, we propose a simple method for more credibly pairing empirical statistical damage estimates derived from recent weather and outcome observations with projected future climate changes and proposed responses. We first analyze agricultural production and loan repayment data from Brazil to understand vulnerability to historical variation in the more predictable components of temperature and rainfall (trend and seasonality) as well as to shocks (both local and over larger spatial scales). This decomposed weather variation over the past two decades explains over 50% of the yield variation in major Brazilian crops and, critically, can be constructed in the same way for future climate projections. Combining our estimates with bias-corrected downscaled climate simulations for Brazil, we find increased variation in yields and revenues (including more bad years and worse outcomes) and higher agricultural loan default at midcentury. Results in this context point to two particularly acute dimensions of vulnerability: Intensified seasonality and local idiosyncratic shocks both contribute to worsening outcomes, along with a reduced capacity for spatially correlated ("covariate") shocks to ameliorate these effects through prices. These findings suggest that resilience strategies should focus on institutions such as water storage, financial services, and reinsurance.