Large Model Uncertainties Research Articles

Large language model uncertainty proxies: discrimination and calibration for medical diagnosis and treatment.

The inability of large language models (LLMs) to communicate uncertainty is a significant barrier to their use in medicine. Before LLMs can be integrated into patient care, the field must assess methods to estimate uncertainty in ways that are useful to physician-users. Evaluate the ability for uncertainty proxies to quantify LLM confidence when performing diagnosis and treatment selection tasks by assessing the properties of discrimination and calibration. We examined confidence elicitation (CE), token-level probability (TLP), and sample consistency (SC) proxies across GPT3.5, GPT4, Llama2, and Llama3. Uncertainty proxies were evaluated against 3 datasets of open-ended patient scenarios. SC discrimination outperformed TLP and CE methods. SC by sentence embedding achieved the highest discriminative performance (ROC AUC 0.68-0.79), yet with poor calibration. SC by GPT annotation achieved the second-best discrimination (ROC AUC 0.66-0.74) with accurate calibration. Verbalized confidence (CE) was found to consistently overestimate model confidence. SC is the most effective method for estimating LLM uncertainty of the proxies evaluated. SC by sentence embedding can effectively estimate uncertainty if the user has a set of reference cases with which to re-calibrate their results, while SC by GPT annotation is the more effective method if the user does not have reference cases and requires accurate raw calibration. Our results confirm LLMs are consistently over-confident when verbalizing their confidence (CE).

The overlooked effects of environmental impacts on root:shoot ratio in experiments and soil-crop models

Process-based soil-crop models are becoming increasingly important to estimate the effects of agricultural management practices and climate change impacts on soil organic carbon (C). Although work has been done on the effects of crop type and climate on the root:shoot (biomass) ratio, there is a gap in research on the effects of specific environmental or management conditions such as drought, temperature, nutrient limitation, elevated CO2 or tillage on the root:shoot ratio and thus, atmospheric C sequestration. In this study, we quantified the effects of these factors on the root:shoot biomass ratio by reviewing the current literature, presented common simulation approaches and performed model simulations using different examples. Finally, we identified different research gaps with respect to the root:shoot ratio with the aim of better estimating and predicting atmospheric C sequestration. A predominantly positive response of the root:shoot ratio was observed in case of elevated CO2 (~12 %), low soil N levels (~44 %), and drought (~14 %). Soil tillage did not affect root:shoot ratio of the major field crops but increased it by ~15 % in case of wheat. There are only few field studies on air temperature increase and the results vary widely (mean − 48 %). The responses of tested models to the mentioned effects root:shoot ratio were slightly positive in case of CO2 elevation (0 to 2 %) and tillage (0 to 8 %), slightly to clearly positive in the case of drought and N limitation depending on the model (1 to 40 %), and very variable in case of the air temperature scenarios. Our study reveals large model uncertainty (especially on temperature effects), particularly for below ground processes that highlight knowledge gaps in simulating root:shoot ratio. We advocate for the need of more model-oriented specific experiments under abiotic stresses to help model improvement. Such research effort would enable more robust and reliable root:shoot ratio simulations.

Winter arctic sea ice volume decline: uncertainties reduced using passive microwave-based sea ice thickness

Arctic sea ice volume (SIV) is a key climate indicator and memory source in sea ice predictions and projections, yet suffering from large observational and model uncertainty. Here, we test whether passive microwave (PMW) data constrain the long-term evolution of Arctic SIV, as recently hypothesized. We find many commonalities in Arctic SIV changes from a PMW sea ice thickness (SIT) 1992-2020 time series reconstructed with a neural network algorithm trained on lidar altimetry, and the reference PIOMAS reanalysis: relatively low differences in SIV mean (4615 km3, 37%), SIV trends (46 km3, 17%), and phased variability (r2=0.55). Key to reduced differences is the consistent evolution of many SIV contributors: seasonal and perennial ice coverage, their SIT contrast, whereas perennial SIT provides the largest remaining uncertainty source. We argue that PMW includes useful SIT information, reducing SIV uncertainty. We foresee progress from sea ice reanalyses combining dynamical models and data assimilation of PMW SIT estimates, in addition to the already assimilated PWM sea ice concentration.

Fuzzy‐PID controller based on improved LFPSO for temperature and humidity control in a CA ripening system

AbstractTemperature and humidity as the key factors affecting the storage and ripening of fruits and vegetables directly determine the quality of fruits and vegetables. In this paper, a temperature and humidity model was constructed based on an integrated device of controlled atmosphere storage and ripening. Aiming at the shortcomings of the temperature and humidity model such as large inertia, nonlinearity, and model uncertainty, a fuzzy‐PID controller based on the improved Lévy flight particle swarm algorithm (LFPSO) is proposed. Initially, a mathematical model of temperature and humidity is developed through mechanism research and parameter estimation. Subsequently, a temperature and humidity fuzzy‐PID control strategy is proposed for regulating temperature and humidity. An improved LFPSO is then introduced to optimize the key parameters of the fuzzy‐PID controller, such as the quantization factor and the scale factor. The superiority of the improved LFPSO algorithm is verified by comparing the test functions. Finally, the improved LFPSO‐fuzzy‐PID controller, fuzzy‐PID controller, and Smith‐PID controller are applied to the simulation model for comparison using the MATLAB Simulink simulation platform. The results show that the improved LFPSO‐fuzzy‐PID control algorithm has a good control effect in the temperature and humidity simulation system of controlled atmosphere (CA) ripening container, which provides a reference for solving the optimization problem of actual engineering design.Practical applicationsIn order to reduce the postharvest cold chain losses of fruits and vegetables, this paper proposes a containerized style device that combines fruits and vegetables storage and ripening for simplifying the losses caused by transshipment at cold chain nodes. Since temperature and humidity play a key role in fruits and vegetables storage and ripening, which directly affect the product quality, this paper establishes a complete mathematical model of CA ripening temperature and humidity system by combining mechanism modeling and parameter identification. In order to keep the temperature and humidity within the ideal range, Smith‐PID controller, fuzzy‐PID controller, and fuzzy‐PID control method based on improved Lévy flight particle swarm optimization are proposed to regulate the air conditioner and humidifier. The performance of three different types of controllers was tested on the MATLAB. The simulation results demonstrate that the improved LFPSO‐fuzzy‐PID controller is superior and more effective than Smith‐PID and fuzzy‐PID controllers.

A multi‐model likelihood analysis of unprecedented extreme rainfall along the east coast of Australia

AbstractA large stretch of the east coast of Australia experienced unprecedented rainfall and flooding over a two‐week period in early 2022. It is difficult to reliably estimate the likelihood of such a rare event from the relatively short observational record, so an alternative is to use data from an ensemble prediction system (e.g., a seasonal or decadal forecast system) to obtain a much larger sample of simulated weather events. This so‐called ‘UNSEEN’ method has been successfully applied in several scientific studies, but those studies typically rely on a single prediction system. In this study, we use data from the Decadal Climate Prediction Project to explore the model uncertainty associated with the UNSEEN method by assessing 10 different hindcast ensembles. Using the 15‐day rainfall total averaged over the river catchments impacted by the 2022 east coast event, we find that the models produce a wide range of likelihood estimates. Even after excluding a number of models that fail basic fidelity tests, estimates of the event return period ranged from 320 to 1814 years. The vast majority of models suggested the event is rarer than a standard extreme value assessment of the observational record (297 years). Such large model uncertainty suggests that multi‐model analysis should become part of the standard UNSEEN procedure.

Enhancing Clear Radiance Generation for Geostationary Hyperspectral Infrared Sounder Using High Temporal Resolution Information

AbstractWith high spatio‐temporal resolution geostationary hyperspectral infrared sounder (GeoHIS) observations, monitoring, and predicting rapidly changing weather events are expected to be improved with continuous information of 3D weather cube. However, due to the nature of radiation, clouds prevent an adequate retrieval of atmospheric thermodynamic information under clear‐sky conditions due to large uncertainties in the current radiative transfer model. Removing cloud effects from GeoHIS sub‐footprint is an alternative approach for enhancing clear radiance generation. Such cloud removal can be achieved through the optimal cloud‐clearing (OCC) method, which usually relies on the assumption of atmospheric spatial homogeneity. With high temporal resolution observations from GeoHIS, cloud removal can also be achieved through OCC but using the assumption of temporal homogeneity. This concept was demonstrated using 15‐min Geostationary Interferometric Infrared Sounder (GIIRS) observations. The longwave GIIRS clear radiances under partially cloud cover can be effectively produced with observation errors less than 0.02 ± 0.86 K and 0.04 ± 0.77 K for 11 and 12 μm, respectively.

Less negative impacts of climate change on crop yields in West Africa in the new CMIP6 climate simulations ensemble

Food insecurity is among one of the greatest risks posed by climate change in Africa, where 90 to 95% of African food production is rainfed and a large proportion of the population already faces chronic hunger and malnutrition. Although, several studies have found robust evidence of future crop yield losses under climate change scenarios, there is wide variation among crops and regions as well as large modeling uncertainties. A large part of this uncertainty stems from climate projections, as climate models may differ in simulating future changes in precipitation and temperature, which could lead to different future crop production scenarios. This work examines the impacts of climate change on crop yields of maize, millet and sorghum in West Africa using climate change projections from the Coupled Model Intercomparison Project 5th Phase (CMIP5) and from the new generation of climate models from the Coupled Model Intercomparison Project 6th Phase (CMIP6). We use the SIMPLACE crop modeling framework to simulate historical and future crop yields, and bootstrap techniques to evaluate projected changes in crop productivity between the CMIP5 and CMIP6 ensembles. Using the new generation of climate models CMIP6, we find that the negative crop yield projections shown by CMIP5 simulations are largely reduced, with even large increases in crop yields when the effect of atmospheric CO2 concentration is considered in the crop model. These differences in crop yield impacts between the CMIP5 and CMIP6 simulations are mainly due to different climate projections of temperature and precipitation in West Africa; CMIP6 projections being significantly wetter and cooler by mid-century and to a lesser extent by the end of the century. Such results highlight the large uncertainties that remain in assessing the impacts of climate change in the region and the consequent difficulty for end-users to anticipate adaptation strategies.

A clustering-based multi-model ensemble projection of near-term precipitation changes over East China and its uncertainty

An ensemble of 28 models from the 6th Coupled Model Intercomparison Project was used to project future changes in annual mean precipitation over East China during 2046–2065 relative to 1995–2014 under the SSP2-4.5 scenario. A precipitation increase of 0.16 mm d−1 is projected over North China by the multi-model ensemble (MME) mean. However, large model uncertainty exists over South China (SC), reducing the fidelity of the MME mean projection. Thus, a clustering-based MME probabilistic projection is presented that projects four possible SC precipitation changes in the future. Precipitation increases are projected in Cluster 1, Cluster 3 and Cluster 4 for 0.51 mm d−1, 0.2 mm d−1 and 0.23 mm d−1, respectively, with occurrence probabilities of 14.3%, 35.7% and 25%, respectively. Conversely, the projected Cluster 2 precipitation decrease is 0.01 mm d−1 with an occurrence probability of 25%. The differences in precipitation change are mainly contributed by dynamic effect due to different circulation changes across clusters. During extended summer, different circulation anomalies over western North Pacific (WNP) among clusters arise from the sea surface temperature anomaly (SSTA) warming patterns over the equatorial central-eastern Pacific, which explain the different precipitation increases over SC. During extended winter, a strong zonal SSTA gradient between the South China Sea and adjacent WNP is projected in Cluster 2, stimulating a zonally vertical cell with anomalous descent over SC and resulting in markedly decreased precipitation. A similar but much weaker zonal SSTA gradient and circulation anomaly are projected in Cluster 3. Distinct meridional SSTA gradients over the WNP are projected in the rest clusters, stimulating shifted descents with a weak effect on SC precipitation.

Robust Higher-Order Spatial Iterative Learning Control for Additive Manufacturing Systems

In this article, a higher-order spatial iterative learning control (HO-SILC) scheme is proposed, targeting heightmap tracking for a class of 3-D structures fabricated by repetitive addition of material in a layer-by-layer fashion using additive manufacturing (AM) technology. AM processes are innately iteration-varying, resulting in large model uncertainties due to iteration-varying system parameters and surface variations. HO-SILC has been shown to be useful in repetitive systems with model uncertainties by improving system performance with respect to convergence speed and robustness. In this article, HO-SILC is used to iteratively construct a feedforward control signal to improve part quality in multilayered AM constructs. The system dynamics are approximated by discrete 2-D spatial convolution kernels that incorporate in-layer and layer-to-layer variations. The proposed HO-SILC framework incorporates data available from previously printed devices, as well as multiple previously printed layers, to enhance the overall performance. The condition for robust monotonic convergence (RMC) of the iteration-varying HO-SILC algorithm is based on the Lyapunov stability criteria. Simulation results of an AM process termed electrohydrodynamic jet (e-jet) printing demonstrate that a well-designed HO-SILC framework is effective and can improve the performance by 60 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\%$</tex-math> </inline-formula> . In addition, HO-SILC is robust to iteration-varying model uncertainties, especially at higher layers where iteration-varying surface variations are more pronounced.

Robust Planning over Restless Groups: Engagement Interventions for a Large-Scale Maternal Telehealth Program

In 2020, maternal mortality in India was estimated to be as high as 130 deaths per 100K live births, nearly twice the UN's target. To improve health outcomes, the non-profit ARMMAN sends automated voice messages to expecting and new mothers across India. However, 38% of mothers stop listening to these calls, missing critical preventative care information. To improve engagement, ARMMAN employs health workers to intervene by making service calls, but workers can only call a fraction of the 100K enrolled mothers. Partnering with ARMMAN, we model the problem of allocating limited interventions across mothers as a restless multi-armed bandit (RMAB), where the realities of large scale and model uncertainty present key new technical challenges. We address these with GROUPS, a double oracle–based algorithm for robust planning in RMABs with scalable grouped arms. Robustness over grouped arms requires several methodological advances. First, to adversarially select stochastic group dynamics, we develop a new method to optimize Whittle indices over transition probability intervals. Second, to learn group-level RMAB policy best responses to these adversarial environments, we introduce a weighted index heuristic. Third, we prove a key theoretical result that planning over grouped arms achieves the same minimax regret--optimal strategy as planning over individual arms, under a technical condition. Finally, using real-world data from ARMMAN, we show that GROUPS produces robust policies that reduce minimax regret by up to 50%, halving the number of preventable missed voice messages to connect more mothers with life-saving maternal health information.

Phytoplankton responses to increasing Arctic river discharge under the present and future climate simulations

In recent decades, the unprecedented rate of Arctic warming has accelerated the high-latitude landmass hydrological cycle, leading to increased river discharge into the Arctic Ocean. This study elucidates the role of Arctic river discharge, which was the large model uncertainty in the Coupled Model Intercomparison Project 6, for the phytoplankton responses in present-day and future climate simulations by adding fresh water into the model. In the present-day climate simulation, additional river discharge decreases the spring phytoplankton biomass. Freshening of Arctic seawater facilitates freezing, increasing sea ice concentration in spring and eventually decreasing phytoplankton due to less availability of light. On the other hand, in the summer, phytoplankton increases due to the surplus of surface nitrate and the increase in the vertical mixing induced by the reduced summer sea ice melting water. In the future climate, the plankton response to the additional freshwater input is similar to the present-day climate. Nevertheless, the major phytoplankton responses are shifted from the Eurasian Basin to the Canada Basin and the East-Siberian Sea, mainly due to the marginal sea ice zone shift from the Barents-Kara Sea to the East Siberian-Chukchi Sea in the future.

The 2018 west-central European drought projected in a warmer climate: how much drier can it get?

Abstract. Projections of changes in extreme droughts under future climate conditions are associated with large uncertainties, owing to the complex genesis of droughts and large model uncertainty in the atmospheric dynamics. In this study we investigate the impact of global warming on soil moisture drought severity in west-central Europe by employing pseudo global warming (PGW) experiments, which project the 1980–2020 period in a globally warmer world. The future analogues of present-day drought episodes allow for investigation of changes in drought severity conditional on the historic day-to-day evolution of the atmospheric circulation. The 2018 west-central European drought is the most severe drought in the 1980–2020 reference period in this region. Under 1.5, 2 and 3 ∘C global warming, this drought episode experiences strongly enhanced summer temperatures but a fairly modest soil moisture drying response compared to the change in climatology. This is primarily because evaporation is already strongly moisture-constrained during present-day conditions, limiting the increase in evaporation and thus the modulation of the temperature response under PGW. Increasing precipitation in winter, spring and autumn limits or prevents an earlier drought onset and duration. Nevertheless, the drought severity, defined as the cumulative soil moisture deficit volume, increases considerably, with 20 % to 39 % under 2 ∘C warming. The extreme drought frequency in the 1980–2020 period strongly increases under 2 ∘C warming. Several years without noticeable droughts under present-day conditions show very strong drying and warming. This results in an increase in 2003-like drought occurrences, compounding with local summer temperature increases considerably above 2 ∘C. Even without taking into account a (potentially large) dynamical response to climate change, drought risk in west-central Europe is strongly enhanced under global warming. Owing to increases in drought frequency, severity and compounding heat, a reduction in recovery times between drought episodes is expected to occur. Our physical climate storyline provides evidence complementing conventional large-ensemble approaches and is intended to contribute to the formulation of effective adaptation strategies.

Anthropogenic climate change exacerbates the risk of successive flood-heat extremes: Multi-model global projections based on the Inter-Sectoral Impact Model Intercomparison Project

The successive flood-heat extreme (SFHE) event, which threatens the securities of human health, economy, and building environment, has attracted extensive research attention recently. However, the potential changes in SFHE characteristics and the global population exposure to SFHE under anthropogenic warming remain unclear. Here, we present a global-scale evaluation of the projected changes and uncertainties in SFHE characteristics (frequency, intensity, duration, land exposure) and population exposure under the Representative Concentration Pathway (RCP) 2.6 and 6.0 scenarios, based on the multi-model ensembles (five global water models forced by four global climate models) within the Inter-Sectoral Impact Model Intercomparison Project 2b framework. The results reveal that, relative to the 1970–1999 baseline period, the SFHE frequency is projected to increase nearly globally by the end of this century, especially in the Qinghai-Tibet Plateau (>20 events/30-year) and the tropical regions (e.g., northern South America, central Africa, and southeastern Asia, >15 events/30-year). The projected higher SFHE frequency is generally accompanied by a larger model uncertainty. By the end of this century, the SFHE land exposure is expected to increase by 12 % (20 %) under RCP2.6 (RCP6.0), and the intervals between flood and heatwave in SFHE tend to decrease by up to 3 days under both RCPs, implying the more intermittent SFHE occurrence under future warming. The SFHE events will lead to the higher population exposure in the Indian Peninsula and central Africa (<10 million person-days) and eastern Asia (<5 million person-days) due to the higher population density and the longer SFHE duration. Partial correlation analysis indicates that the contribution of flood to the SFHE frequency is greater than that of heatwave for most global regions, but the SFHE frequency is dominated by the heatwave in northern North America and northern Asia.

Satellites reveal Earth's seasonally shifting dust emission sources

Establishing mineral dust impacts on Earth's systems requires numerical models of the dust cycle. Differences between dust optical depth (DOD) measurements and modelling the cycle of dust emission, atmospheric transport, and deposition of dust indicate large model uncertainty due partially to unrealistic model assumptions about dust emission frequency. Calibrating dust cycle models to DOD measurements typically in North Africa, are routinely used to reduce dust model magnitude. This calibration forces modelled dust emissions to match atmospheric DOD but may hide the correct magnitude and frequency of dust emission events at source, compensating biases in other modelled processes of the dust cycle. Therefore, it is essential to improve physically based dust emission modules.Here we use a global collation of satellite observations from previous studies of dust emission point source (DPS) dichotomous frequency data. We show that these DPS data have little-to-no relation with MODIS DOD frequency. We calibrate the albedo-based dust emission model using the frequency distribution of those DPS data. The global dust emission uncertainty constrained by DPS data (±3.8 kg m−2 y−1) provides a benchmark for dust emission model development. Our calibrated model results reveal much less global dust emission (29.1 ± 14.9 Tg y−1) than previous estimates, and show seasonally shifting dust emission predominance within and between hemispheres, as opposed to a persistent North African dust emission primacy widely interpreted from DOD measurements.Earth's largest dust emissions, proceed seasonally from East Asian deserts in boreal spring, to Middle Eastern and North African deserts in boreal summer and then Australian shrublands in boreal autumn-winter. This new analysis of dust emissions, from global sources of varying geochemical properties, have far-reaching implications for current and future dust-climate effects. For more reliable coupled representation of dust-climate projections, our findings suggest the need to re-evaluate dust cycle modelling and benefit from the albedo-based parameterisation.

Uncertainty-driven dynamics for active learning of interatomic potentials

Machine learning (ML) models, if trained to data sets of high-fidelity quantum simulations, produce accurate and efficient interatomic potentials. Active learning (AL) is a powerful tool to iteratively generate diverse data sets. In this approach, the ML model provides an uncertainty estimate along with its prediction for each new atomic configuration. If the uncertainty estimate passes a certain threshold, then the configuration is included in the data set. Here we develop a strategy to more rapidly discover configurations that meaningfully augment the training data set. The approach, uncertainty-driven dynamics for active learning (UDD-AL), modifies the potential energy surface used in molecular dynamics simulations to favor regions of configuration space for which there is large model uncertainty. The performance of UDD-AL is demonstrated for two AL tasks: sampling the conformational space of glycine and sampling the promotion of proton transfer in acetylacetone. The method is shown to efficiently explore the chemically relevant configuration space, which may be inaccessible using regular dynamical sampling at target temperature conditions.

Reinforcement Learning for Autonomous Underwater Vehicles via Data-Informed Domain Randomization

Autonomous Underwater Vehicles (AUVs) or underwater vehicle-manipulator systems often have large model uncertainties from degenerated or damaged thrusters, varying payloads, disturbances from currents, etc. Other constraints, such as input dead zones and saturations, make the feedback controllers difficult to tune online. Model-free Reinforcement Learning (RL) has been applied to control AUVs, but most results were validated through numerical simulations. The trained controllers often perform unsatisfactorily on real AUVs; this is because the distributions of the AUV dynamics in numerical simulations and those of real AUVs are mismatched. This paper presents a model-free RL via Data-informed Domain Randomization (DDR) for controlling AUVs, where the mismatches between the trajectory data from numerical simulations and the real AUV were minimized by adjusting the parameters in the simulated AUVs. The DDR strategy extends the existing adaptive domain randomization technique by aggregating an input network to learn mappings between control signals across domains, enabling the controller to adapt to sudden changes in dynamics. The proposed RL via DDR was tested on the problems of AUV pose regulation through extensive numerical simulations and experiments in a lab tank with an underwater positioning system. These results have demonstrated the effectiveness of RL-DDR for transferring trained controllers to AUVs with different dynamics.

Navigare necesse est: markets, crises, and our models

The events of the past decade and a half have posed unprecedented challenges to macroeconomic policymaking. Modern policymaking has long been relying on a spectrum of models both at national and supranational levels. However, none of the standard models have predicted any of the major crises of the past decade and a half, nor could they capture the dynamics of the ensuing readjustment processes well. In general, standard workhorse models make several assumptions that may not hold in the economies they are applied to. Moreover, they perform particularly poorly during crisis episodes. This short article reviews the issues surrounding these models and proposes a simple modelling strategy to be applied in the face of such large model uncertainty.

Simulated Cycles of East Asian Temperature and Precipitation Over the Past 425 ka

AbstractRecords from a wide range of geological archives covering the last few glacial‐interglacial cycles show large inconsistencies in the East Asian summer monsoon variability, which severely hampers our understanding of the evolution and potential mechanisms of the regional East Asian climate on orbital timescales. Here, we examine the simulated temperature and precipitation in East Asia based on a series of equilibrium simulations conducted for the past 425 ka, and we investigate the sensitivity of temperature and precipitation to potential forcings. Our simulations show that, in East Asia, the seasonal mean temperature is dominated by a ∼20‐kyr cycle, and the annual mean temperature (AMT) is dominated by a ∼100‐kyr cycle, which is consistent with previous modeling efforts and geological reconstructions. Additional sensitivity experiments indicate that the greenhouse gas concentration, in combination with the ice volume, is the dominant force for the variations of AMT in East Asia on orbital timescales. For the precipitation in East Asia, our equilibrium simulations and additional sensitivity experiments, together with comprehensive model‐data intercomparison analysis, suggest that the cycles of simulated annual mean precipitation over East Asia are highly model‐dependent, although the dominant ∼20‐kyr cycle in summer precipitation appears to be a robust feature. Overall, the results highlight the large model uncertainty with regard to the relative roles of forcings in hydroclimate variations in East Asia on orbital time scales. There is, therefore, an urgent need to implement more realistic precipitation schemes in models in order to decrease the model spread in simulated precipitation.

Assessment of the benefits of climate model weights for ensemble analysis in three urban precipitation frequency studies

AbstractIn hydrology, projected climate change impact assessment studies typically rely on ensembles of downscaled climate model outputs. Due to large modeling uncertainties, the ensembles are often averaged to provide a basis for studying the effects of climate change. A key issue when analyzing averages of a climate model ensemble is whether to weight all models in the ensemble equally, often referred to as the equal‐weights or unweighted approach, or to use a weighted approach, where, in general, each model would have a different weight. Many studies have advocated for the latter, based on the assumption that models that are better at simulating the past, that is, the models with higher hindcast accuracy, will give more accurate forecasts for the future and thus should receive higher weights. To examine this issue, observed and modeled daily precipitation frequency (PF) estimates for three urban areas in the United States, namely Boston, Massachusetts; Houston, Texas; and Chicago, Illinois, were analyzed. The comparison used the raw output of 24 Coupled Model Intercomparison Project Phase 5 (CMIP5) models. The PFs from these models were compared with the observed PFs for a specific historical training period to determine model weights for each area. The unweighted and weighted averaged model PFs from a more recent testing period were then compared with their corresponding observed PFs to determine if weights improved the estimates. These comparisons indeed showed that the weighted averages were closer to the observed values than the unweighted averages in nearly all cases. The study also demonstrated how weights can help reduce model spread in future climate projections by comparing the unweighted and weighted ensemble standard deviations in these projections. In all studied scenarios, the weights actually reduced the standard deviations compared to the equal‐weights approach. Finally, an analysis of the results' sensitivity to the areal reduction factor used to allow comparisons between point station measurements and grid‐box averages is provided.

Attribution of Stratospheric and Tropospheric Ozone Changes Between 1850 and 2014 in CMIP6 Models

AbstractWe quantify the impacts of halogenated ozone‐depleting substances (ODSs), greenhouse gases (GHGs), and short‐lived ozone precursors on ozone changes between 1850 and 2014 using single‐forcing perturbation simulations from several Earth system models with interactive chemistry participating in the Coupled Model Intercomparison Project Aerosol and Chemistry Model Intercomparison Project. We present the responses of ozone to individual forcings and an attribution of changes in ozone columns and vertically resolved stratospheric and tropospheric ozone to these forcings. We find that whilst substantial ODS‐induced ozone loss dominates the stratospheric ozone changes since the 1970s, in agreement with previous studies, increases in tropospheric ozone due to increases in short‐lived ozone precursors and methane since the 1950s make increasingly important contributions to total column ozone (TCO) changes. Increases in methane also lead to substantial extra‐tropical stratospheric ozone increases. Impacts of nitrous oxide and carbon dioxide on stratospheric ozone are significant but their impacts on TCO are small overall due to several opposing factors and are also associated with large dynamical variability. The multi‐model mean (MMM) results show a clear change in the stratospheric ozone trends after 2000 due to now declining ODSs, but the trends are generally not significantly positive, except in the extra‐tropical upper stratosphere, due to relatively small changes in forcing over this period combined with large model uncertainty. Although the MMM ozone compares well with the observations, the inter‐model differences are large primarily due to the large differences in the models' representation of ODS‐induced ozone depletion.