Synthetic Ensembles Research Articles

Internal Versus Forced Variability Metrics for General Circulation Models Using Information Theory

AbstractOcean model simulations show variability due to intrinsic chaos and external forcing (air‐sea fluxes, river input, etc.). It is important to estimate their contributions to total variability for attribution. Using variance to estimate variability might be unreliable due to non‐Gaussian higher statistical moments. We show the use of non‐parametric information theory metrics, Shannon entropy and mutual information, for measuring internal and forced variability in ocean models. These metrics are applied to spatially and temporally averaged data. The metrics delineate relative intrinsic to total variability in a wider range of circumstances than previous approaches based on variance ratios. The metrics are applied to (a) a synthetic ensemble of random vectors, (b) ocean component of a global climate (GFDL‐ESM2M) large ensemble, (c) ensemble of a realistic coastal ocean model. The information theory metric qualitatively agrees with the variance‐based metric and possibly identifies regions of nonlinear correlations. In application (2)–the climate ensemble–the information theory metric detects higher temperature intrinsic variability in the Arctic region compared to the variance metric illustrating that the former is robust in a skewed probability distribution (Arctic sea surface temperature) resulting from sharply nonlinear behavior (freezing point). In application (3)–coastal ensemble–variability is dominated by external forcing. Using different selective forcing ensembles, we quantify the sensitivity of the coastal model to different types of external forcing: variations in the river runoff and changes in wind product do not add information (i.e., variability) during summer. Information theory enables ranking how much each forcing type contributes across multiple variables.

Generative Ensemble Deep Learning Severe Weather Prediction from a Deterministic Convection-Allowing Model

Abstract An ensemble postprocessing method is developed for the probabilistic prediction of severe weather (tornadoes, hail, and wind gusts) over the conterminous United States (CONUS). The method combines conditional generative adversarial networks (CGANs), a type of deep generative model, with a convolutional neural network (CNN) to postprocess convection-allowing model (CAM) forecasts. The CGANs are designed to create synthetic ensemble members from deterministic CAM forecasts, and their outputs are processed by the CNN to estimate the probability of severe weather. The method is tested using High-Resolution Rapid Refresh (HRRR) 1–24-h forecasts as inputs and Storm Prediction Center (SPC) severe weather reports as targets. The method produced skillful predictions with up to 20% Brier skill score (BSS) increases compared to other neural-network-based reference methods using a testing dataset of HRRR forecasts in 2021. For the evaluation of uncertainty quantification, the method is overconfident but produces meaningful ensemble spreads that can distinguish good and bad forecasts. The quality of CGAN outputs is also evaluated. Results show that the CGAN outputs behave similarly to a numerical ensemble; they preserved the intervariable correlations and the contribution of influential predictors as in the original HRRR forecasts. This work provides a novel approach to postprocess CAM output using neural networks that can be applied to severe weather prediction. Significance Statement We use a new machine learning (ML) technique to generate probabilistic forecasts of convective weather hazards, such as tornadoes and hailstorms, with the output from high-resolution numerical weather model forecasts. The new ML system generates an ensemble of synthetic forecast fields from a single forecast, which are then used to train ML models for convective hazard prediction. Using this ML-generated ensemble for training leads to improvements of 10%–20% in severe weather forecast skills compared to using other ML algorithms that use only output from the single forecast. This work is unique in that it explores the use of ML methods for producing synthetic forecasts of convective storm events and using these to train ML systems for high-impact convective weather prediction.

Evaluating Contour Band Depth as a Method for Understanding Ensemble Uncertainty

Abstract Estimating and predicting the state of the atmosphere is a probabilistic problem for which an ensemble modeling approach often is taken to represent uncertainty in the system. Common methods for examining uncertainty and assessing performance for ensembles emphasize pointwise statistics or marginal distributions. However, these methods lose specific information about individual ensemble members. This paper explores contour band depth (cBD), a method of analyzing uncertainty in terms of contours of scalar fields. cBD is fully nonparametric and induces an ordering on ensemble members that leads to box-and-whisker-plot-type visualizations of uncertainty for two-dimensional data. By applying cBD to synthetic ensembles, we demonstrate that it provides enhanced information about the spatial structure of ensemble uncertainty. We also find that the usefulness of the cBD analysis depends on the presence of multiple modes and multiple scales in the ensemble of contours. Finally, we apply cBD to compare various convection-permitting forecasts from different ensemble prediction systems and find that the value it provides in real-world applications compared to standard analysis methods exhibits clear limitations. In some cases, contour boxplots can provide deeper insight into differences in spatial characteristics between the different ensemble forecasts. Nevertheless, identification of outliers using cBD is not always intuitive, and the method can be especially challenging to implement for flow that exhibits multiple spatial scales (e.g., discrete convective cells embedded within a mesoscale weather system). Significance Statement Predictions of Earth’s atmosphere inherently come with some degree of uncertainty owing to incomplete observations and the chaotic nature of the system. Understanding that uncertainty is critical when drawing scientific conclusions or making policy decisions from model predictions. In this study, we explore a method for describing model uncertainty when the quantities of interest are well represented by contours. The method yields a quantitative visualization of uncertainty in both the location and the shape of contours to an extent that is not possible with standard uncertainty quantification methods and may eventually prove useful for the development of more robust techniques for evaluating and validating numerical weather models.

Modeled Interannual Variability of Arctic Sea Ice Cover is within Observational Uncertainty

Abstract Internal variability is the dominant cause of projection uncertainty of Arctic sea ice in the short and medium term. However, it is difficult to determine the realism of simulated internal variability in climate models, as observations only provide one possible realization while climate models can provide numerous different realizations. To enable a robust assessment of simulated internal variability of Arctic sea ice, we use a resampling technique to build synthetic ensembles for both observations and climate models, focusing on interannual variability, which is the dominant time scale of Arctic sea ice internal variability. We assess the realism of the interannual variability of Arctic sea ice cover as simulated by six models from phase 5 of the Coupled Model Intercomparison Project (CMIP5) that provide large ensembles compared to four observational datasets. We augment the standard definition of model and observational consistency by representing the full distribution of resamplings, analogous to the distribution of variability that could have randomly occurred. We find that modeled interannual variability typically lies within observational uncertainty. The three models with the smallest mean state biases are the only ones consistent in the pan-Arctic for all months, but no model is consistent for all regions and seasons. Hence, choosing the right model for a given task as well as using internal variability as an additional metric to assess sea ice simulations is important. The fact that CMIP5 large ensembles broadly simulate interannual variability consistent within observational uncertainty gives confidence in the internal projection uncertainty for Arctic sea ice based on these models. Significance Statement The purpose of this study is to evaluate the historical simulated internal variability of Arctic sea ice in climate models. Determining model realism is important to have confidence in the projected sea ice evolution from these models, but so far only mean state and trends are commonly assessed metrics. Here we assess internal variability with a focus on the interannual variability, which is the dominant time scale for internal variability. We find that, in general, models agree well with observations, but as no model is within observational uncertainty for all months and locations, choosing the right model for a given task is crucial. Further refinement of internal variability realism assessments will require reduced observational uncertainty.

Alternate Histories: Synthetic Large Ensembles of Sea‐Air CO2 Flux

AbstractWe use a statistical emulation technique to construct synthetic ensembles of global and regional sea‐air carbon dioxide (CO2) flux from four observation‐based products over 1985–2014. Much like ensembles of Earth system models that are constructed by perturbing their initial conditions, our synthetic ensemble members exhibit different phasing of internal variability and a common externally forced signal. Our synthetic ensembles illustrate an important role for internal variability in the temporal evolution of global and regional CO2 flux and produce a wide range of possible trends over 1990–1999 and 2000–2009. We assume a specific externally forced signal and calculate the rank of the observed trends within the distribution of statistically modeled synthetic trends during these periods. Over the decade 1990–1999, three of four observation‐based products exhibit small negative trends in globally integrated sea‐air CO2 flux (i.e., enhanced ocean CO2 absorption with time) that are within one standard deviation of the mean in their respective synthetic ensembles. Over the decade 2000–2009, however, three products show large negative trends in globally integrated sea‐air CO2 flux that have a low rate of occurrence in their synthetic ensembles. The largest positive trends in global and Southern Ocean flux over 1990–1999 and the largest negative trends over 2000–2009 fall nearly two standard deviations away from the mean in their ensembles. Our approach provides a new perspective on the important role of internal variability in sea‐air CO2 flux trends, and furthers understanding of the role of internal and external processes in driving observed sea‐air CO2 flux variability.

Estimation of Hydropower Potential Using Bayesian and Stochastic Approaches for Streamflow Simulation and Accounting for the Intermediate Storage Retention

Hydropower is the most widely used renewable power source worldwide. The current work presents a methodological tool to determine the hydropower potential of a reservoir based on available hydrological information. A Bayesian analysis of the river flow process and of the reservoir water volume is applied, and the estimated probability density function parameters are integrated for a stochastic analysis and long-term simulation of the river flow process, which is then used as input for the water balance in the reservoir, and thus, for the estimation of the hydropower energy potential. The stochastic approach is employed in terms of the Monte Carlo ensemble technique in order to additionally account for the effect of the intermediate storage retention due to the thresholds of the reservoir. A synthetic river flow timeseries is simulated by preserving the marginal probability distribution function properties of the observed timeseries and also by explicitly preserving the second-order dependence structure of the river flow in the scale domain. The synthetic ensemble is used for the simulation of the reservoir water balance, and the estimation of the hydropower potential is used for covering residential energy needs. For the second-order dependence structure of the river flow, the climacogram metric is used. The proposed methodology has been implemented to assess different reservoir volume scenarios offering the associated hydropower potential for a case study at the island of Crete in Greece. The tool also provides information on the probability of occurrence of the specific volumes based on available hydrological data. Therefore, it constitutes a useful and integrated framework for evaluating the hydropower potential of any given reservoir. The effects of the intermediate storage retention of the reservoir, the marginal and dependence structures of the parent distribution of inflow and the final energy output are also discussed.

Comparison of two model predictive control methods that can deal with ensemble forecasts

In this work we numerically implement tree-based model predictive control (TB-MPC) to a Dutch water system. The TB-MPC is compared to deterministic control (MPC) and ensemble model predictive control (ENS-MPC). The comparison is done using synthetic ensembles with different dispersions. For example, in the deterministic runs, members from the ensembles with different percentiles are used. In total 44 closed-loop experiments were performed: the control action is calculated based on a linear model and sent to a linear model combined with non-linear pre-processing as simulation model. Using ensembles leads to a more robust control of the water system as ensemble approaches are able to control the water system even in case of uncertain forecast. In this case study TB-MPC shows better performance for larger uncertainties while ENS-MPC performs better for smaller uncertainties. This can be beneficial when extreme events are expected, but for day-to-day forecast TB-MPC shows a great potential.

The inherent uncertainty of precipitation variability, trends, and extremes due to internal variability, with implications for Western US water resources

AbstractThe approximately century-long instrumental record of precipitation over land reflects a single sampling of internal variability. Thus, the spatiotemporal evolution of the observations is only one realization of `what could have occurred' given the same climate system and boundary conditions, but different initial conditions. While climate models can be used to produce initial-condition large ensembles that explicitly sample different sequences of internal variability, an analogous approach is not possible for the real world. Here, we explore the use of a statistical model for monthly precipitation to generate synthetic ensembles based on a single record. When tested within the context of the NCAR Community Earth System Model version 1 Large Ensemble (CESM1-LE), we find that the synthetic ensemble can closely reproduce the spatiotemporal statistics of variability and trends in winter precipitation over the extended contiguous United States, and that it is difficult to infer the climate change signal in a single record given the magnitude of the variability. We additionally create a synthetic ensemble based on the Global Precipitation Climatology Centre (GPCC) dataset, termed the GPCC-synth-LE; comparison of the GPCC-synth-LE with the CESM1-based ensembles reveals differences in the spatial structures and magnitudes of variability, highlighting the advantages of an observationally-based ensemble. We finally use the GPCC-synth-LE to analyze three water resource metrics in the Upper Colorado River Basin: frequency of dry, wet, and whiplash years. Thirty-one year ‘climatologies’ in the GPCC-synth-LE can differ by over 20% in these key water resource metrics due to sampling of internal variability, and individual ensemble members in the GPCC-synth-LE can exhibit large near-monotonic trends over the course of the last century due to sampling of variability alone.

Alternate History: A Synthetic Ensemble of Ocean Chlorophyll Concentrations

AbstractInternal climate variability plays an important role in the abundance and distribution of phytoplankton in the global ocean. Previous studies using large ensembles of Earth system models (ESMs) have demonstrated their utility in the study of marine phytoplankton variability. These ESM large ensembles simulate the evolution of multiple alternate realities, each with a different phasing of internal climate variability. However, ESMs may not accurately represent real world variability as recorded via satellite and in situ observations of ocean chlorophyll over the past few decades. Observational records of surface ocean chlorophyll equate to a single ensemble member in the large ensemble framework, and this can cloud the interpretation of long‐term trends: are they externally forced, caused by the phasing of internal variability, or both? Here, we use a novel statistical emulation technique to place the observational record of surface ocean chlorophyll into the large ensemble framework. Much like a large initial condition ensemble generated with an ESM, the resulting synthetic ensemble represents multiple possible evolutions of ocean chlorophyll concentration, each with a different sampling of internal climate variability. We further demonstrate the validity of our statistical approach by recreating an ESM ensemble of chlorophyll using only a single ESM ensemble member. We use the synthetic ensemble to explore the interpretation of long‐term trends in the presence of internal variability and find a wider range of possible trends in chlorophyll due to the sampling of internal variability in subpolar regions than in subtropical regions.

Finding the Fingerprint of Anthropogenic Climate Change in Marine Phytoplankton Abundance

We review how phytoplankton abundance may be responding to the increase in stratification associated with anthropogenic climate change, providing context on the utility of remote sensing datasets and Earth system model output to understand these perturbations. Assessing disruption in the ocean biosphere using remote sensing datasets is challenged by the relatively short length of the observational record, restricting our ability to disentangle fluctuations due to internal climate variability from those imposed by externally forced anthropogenic trends. Ensembles of Earth system models can be used to quantify past and future drivers, but may not skillfully predict observed spatial patterns and temporal dynamics in marine phytoplankton. To better understand the role of internal climate variability in the observational record, we construct a synthetic ensemble of global chlorophyll concentration over the MODIS satellite mission using statistical emulation techniques. We emphasize the use of a synthetic ensemble to illuminate the role of internal climate variability in the evolution of the ocean biosphere over time.

Using Natural Experiments and Counterfactuals for Causal Assessment: River Salinity and the Ganges Water Agreement

AbstractThe effect of environmental policy on water resources is often challenging to evaluate due to dynamic interactions between people and water, particularly in data‐scarce watersheds. Increasing interactions between society and hydrology present a need to understand causal relations for improved assessment and prediction in complex human‐water systems. Conventional approaches to causal assessment in hydrology are sometimes insufficient due to data scarcity or system complexity. We argue that natural experiments present a promising and complementary avenue for assessing causal relations in such systems. In this spirit, we describe a natural experiment to assess causal effects of the Ganges water treaty, between India and Bangladesh, on streamflow and channel salinity in the Ganges delta in Bangladesh. We apply causal inference to assess the effect of the treaty on streamflow and use the results to generate synthetic ensembles of streamflow and salinity under a realistic scenario (with the treaty) and a counterfactual scenario (without the treaty). We then use synthetic streamflow ensembles to model salinity ensembles. The Ganges water treaty increased dry‐season streamflow in Bangladesh by approximately 18% and decreased channel salinity by approximately 10%. The treaty has the greatest effect on salinity in Bangladesh in the driest years, but the overall effect is small compared with natural variability. We show that our approach accounts for natural hydrologic variability to accurately assess the causal effect of the treaty, compared with a naive approach that greatly overestimates the effect. This research demonstrates the value of natural experiments for causal assessment in coupled human‐water systems.

A Simplified Assimilation Scheme for a Coastal Wave Model Using Concepts of Particle Filter

In this work, observations of ocean wave height from satellite altimeters are assimilated into the coastal wave model simulating waves nearshore (SWAN) operating in the Indian coastal waters. The study has two distinctive features. The most important is the use of certain concepts of the modern particle filter technique, which does not represent the model probability density function (PDF) by a Gaussian. The other feature is the joint assimilation of data from three altimeters. The method starts by generating an initial ensemble by the bootstrap technique in which the significant wave height (SWH) field of a control run is perturbed by randomly adding bias to produce a member of the ensemble. At the first assimilation time, a weight-based resampling of the individual members, known as particles, is performed. Stronger particles are retained, while the weaker ones are discarded. In order to keep the ensemble size constant, the algorithm replicates a few strong members. After this resampling, a single model run is employed in the forecast step using a kind of averaging. At the next assimilation time, a synthetic ensemble is again formed by the same bootstrapping, and observations are assimilated. The forecast–assimilation cycle is repeated until the observations are exhausted. Assimilation experiments were conducted for 6 months from February through July in 2016. The power of the technique is evaluated by validating results with altimeter data as well as independent data sets from moored buoys. The results are found to be extremely encouraging for the use of this method in carrying out coastal wave forecasting.

Uncertainty component estimates in transient climate projections

Quantifying model uncertainty and internal variability components in climate projections has been paid a great attention in the recent years. For multiple synthetic ensembles of climate projections, we compare the precision of uncertainty component estimates obtained respectively with the two Analysis of Variance (ANOVA) approaches mostly used in recent works: the popular Single Time approach (STANOVA), based on the data available for the considered projection lead time and a time series based approach (QEANOVA), which assumes quasi-ergodicity of climate outputs over the available simulation period. We show that the precision of all uncertainty estimates is higher when more members are used, when internal variability is smaller and/or the response-to-uncertainty ratio is higher. QEANOVA estimates are much more precise than STANOVA ones: QEANOVA simulated confidence intervals are roughly 3–5 times smaller than STANOVA ones. Except for STANOVA when less than three members is available, the precision is rather high for total uncertainty and moderate for internal variability estimates. For model uncertainty or response-to-uncertainty ratio estimates, the precision is low for QEANOVA to very low for STANOVA. In the most unfavorable configurations (small number of members, large internal variability), large over- or underestimation of uncertainty components is thus very likely. In a number of cases, the uncertainty analysis should thus be preferentially carried out with a time series approach or with a local-time series approach, applied to all predictions available in the temporal neighborhood of the target prediction lead time.

A flexible additive inflation scheme for treating model error in ensemble Kalman filters

Data assimilation algorithms require an accurate estimate of the uncertainty of the prior (background) field that cannot be adequately represented by the ensemble of numerical model simulations. Partially, this is due to the sampling error that arises from the use of a small number of ensemble members to represent the background‐error covariance. It is also partially a consequence of the fact that the geophysical model does not represent its own error. Several mechanisms have been introduced so far to alleviate the detrimental effects of misrepresented ensemble covariances, allowing for the successful implementation of ensemble data assimilation techniques for atmospheric dynamics. One of the established approaches is additive inflation, which consists of perturbing each ensemble member with a sample from a given distribution. This results in a fixed rank of the effective model‐error covariance matrix. In this article, a more flexible approach is introduced, where the model error samples are treated as additional synthetic ensemble members, which are used in the update step of data assimilation but are not forecast. This way, the rank of the model‐error covariance matrix can be chosen independently of the ensemble. The effect of this altered additive inflation method on the performance of the filter is analyzed here in an idealized experiment. It is shown that the additional synthetic ensemble members can make it feasible to achieve convergence in an otherwise divergent parameter setting of data assimilation. The use of this method also allows for a less stringent localization radius.

Internal Variability and Regional Climate Trends in an Observational Large Ensemble

Recent observed climate trends result from a combination of external radiative forcing and internally generated variability. To better contextualize these trends and forecast future ones, it is necessary to properly model the spatiotemporal properties of the internal variability. Here, a statistical model is developed for terrestrial temperature and precipitation, and global sea level pressure, based upon monthly gridded observational datasets that span 1921–2014. The model is used to generate a synthetic ensemble, each member of which has a unique sequence of internal variability but with statistical properties similar to the observational record. This synthetic ensemble is combined with estimates of the externally forced response from climate models to produce an observational large ensemble (OBS-LE). The 1000 members of the OBS-LE display considerable diversity in their 50-yr regional climate trends, indicative of the importance of internal variability on multidecadal time scales. For example, unforced atmospheric circulation trends associated with the northern annular mode can induce winter temperature trends over Eurasia that are comparable in magnitude to the forced trend over the past 50 years. Similarly, the contribution of internal variability to winter precipitation trends is large across most of the globe, leading to substantial regional uncertainties in the amplitude and, in some cases, the sign of the 50-yr trend. The OBS-LE provides a real-world counterpart to initial-condition model ensembles. The approach could be expanded to using paleo-proxy data to simulate longer-term variability.

Bayesian estimation of extreme flood quantiles using a rainfall-runoff model and a stochastic daily rainfall generator

Extreme flood estimation has been a key research topic in hydrological sciences. Reliable estimates of such events are necessary as structures for flood conveyance are continuously evolving in size and complexity and, as a result, their failure-associated hazards become more and more pronounced. Due to this fact, several estimation techniques intended to improve flood frequency analysis and reducing uncertainty in extreme quantile estimation have been addressed in the literature in the last decades. In this paper, we develop a Bayesian framework for the indirect estimation of extreme flood quantiles from rainfall-runoff models. In the proposed approach, an ensemble of long daily rainfall series is simulated with a stochastic generator, which models extreme rainfall amounts with an upper-bounded distribution function, namely, the 4-parameter lognormal model. The rationale behind the generation model is that physical limits for rainfall amounts, and consequently for floods, exist and, by imposing an appropriate upper bound for the probabilistic model, more plausible estimates can be obtained for those rainfall quantiles with very low exceedance probabilities. Daily rainfall time series are converted into streamflows by routing each realization of the synthetic ensemble through a conceptual hydrologic model, the Rio Grande rainfall-runoff model. Calibration of parameters is performed through a nonlinear regression model, by means of the specification of a statistical model for the residuals that is able to accommodate autocorrelation, heteroscedasticity and nonnormality. By combining the outlined steps in a Bayesian structure of analysis, one is able to properly summarize the resulting uncertainty and estimating more accurate credible intervals for a set of flood quantiles of interest. The method for extreme flood indirect estimation was applied to the American river catchment, at the Folsom dam, in the state of California, USA. Results show that most floods, including exceptionally large non-systematic events, were reasonably estimated with the proposed approach. In addition, by accounting for uncertainties in each modeling step, one is able to obtain a better understanding of the influential factors in large flood formation dynamics.

Improving Ensemble-Based Quantitative Precipitation Forecasts for Topography-Enhanced Typhoon Heavy Rainfall over Taiwan with a Modified Probability-Matching Technique

Abstract In this paper, a modified probability-matching technique is developed for ensemble-based quantitative precipitation forecasts (QPFs) associated with landfalling typhoons over Taiwan. The main features of this technique include a resampling of the ensemble realizations, a rainfall pattern adjustment, and a bias correction. Using this technique, a synthetic ensemble is created for the purpose of rainfall prediction from a large-size (32 members), low-resolution (36 km) ensemble and a small-size (8 members), high-resolution (4 km) ensemble. The rainfall pattern is adjusted based on the precipitation distribution of the 36- and 4-km ensembles. A bias-correction scheme is then applied to remove the known systematic bias from the resampled 4-km ensemble realizations as part of the probability-matching procedure. The modified probability-matching scheme is shown to substantially reduce or eliminate the intrinsic model rainfall bias and to provide better QPF guidance. The encouraging results suggest that this modified probability-matching technique is a useful tool for the QPF of the topography-enhanced typhoon heavy rainfall over Taiwan using ensemble forecasts at dual resolutions.

Critical Assessment of Paramagnetic Relaxation Enhancement for Ensemble Calculation of Disordered States

In NMR spectroscopy, paramagnetic relaxation enhancement (PRE) can be used to probe transiently formed long-range contacts in macromolecular structures. This is particularly powerful for the structural characterization of intrinsically disordered and unfolded proteins. Here we assess the structural information encoded in PRE data, and its use in describing ensemble representations of disordered states. We seek to determine both the optimal number of PRE labels for structural determination as well as their capabilities in reproducing contact regions and contact populations. We first constructed several ensembles of a model protein with a variety of imposed tertiary contacts, and then attempted to reproduce the ensembles through simulations restrained by theoretical PRE intensities. Almost universally, the PRE data is able to recapture the contacts of the synthetic ensembles provided that one PRE label is placed every 10-20 residues. However, the degeneracy inherent in ensemble-averaged PRE data, compounded by the radius to the minus sixth dependency, makes it difficult to distinguish between contact distances and populations. As a consequence, while the contacts themselves remain visible, their populations are largely determined by the size of the simulated ensemble.