Ensemble Data Research Articles

Sequential model identification with reversible jump ensemble data assimilation method

In data assimilation (DA) schemes, the form representing the processes in the evolution models are pre-determined except some parameters to be estimated. In some applications, such as the contaminant solute transport model and the gas reservoir model, the modes in the equations within the evolution model cannot be predetermined from the outset and may change with the time. We propose a framework of sequential DA method named Reversible Jump Ensemble Filter (RJEnF) to identify the governing modes of the evolution model over time. The main idea is to introduce the Reversible Jump Markov Chain Monte Carlo (RJMCMC) method to the DA schemes to fit the situation where the modes of the evolution model are unknown and the dimension of the parameters is changing. Our framework allows us to identify the modes in the evolution model and their changes, as well as estimate the parameters and states of the dynamic system. Numerical experiments are conducted and the results show that our framework can effectively identify the underlying evolution models and increase the predictive accuracy of DA methods.

Investigating the mechanisms of an intense coastal rainfall event during TAHOPE/PRECIP-IOP3 using a multiscale radar ensemble data assimilation system

Abstract The joint Taiwan-Area Heavy Rain Observation and Prediction Experiment (TAHOPE)/Prediction of Rainfall Extremes Campaign In the Pacific (PRECIP) field campaign between Taiwan and the United States took place from late May to mid-August in 2022. The field campaign aimed to understand the dynamics, thermodynamics, and predictability of heavy rainfall events in the Taiwan area. This study investigated the mechanisms of a heavy rainfall event that occurred on 6–7 June during the intensive observation period-3 (IOP3) of the field campaign. Heavy rainfall occurs on Taiwan’s western coast when a Meiyu front hovers in northern Taiwan. A multiscale radar ensemble data assimilation system based on the successive covariance localization (SCL) method is used to derive a high-resolution analysis for forecasts. Two numerical experiments are conducted with the use of convective-scale (RDA) or multiscale (MRDA) corrections in the assimilation of the radial velocity from operational radars at Chigu and Wufen, and the additional S-Pol radar deployed at Hsinchu during the field campaign. Compared with RDA, MRDA results in large-area wind corrections, which help reshape and relocate a low-level mesoscale vortex, a key element of this heavy rainfall event, offshore of western central Taiwan and enhances the front intensity offshore of northwestern Taiwan. Consequently, MRDA improves the 6-h heavy rainfall prediction over the coast of western Taiwan and better represents the elongated rainband in northern Taiwan during the 3- to 6-h forecast. Sensitivity experiments demonstrate the importance of assimilating winds from Chigu and S-Pol radar in establishing low-level mesoscale vortex and convergence zones.

A novel statistical framework of drought projection by improving ensemble future climate model simulations under various climate change scenarios.

Unlike other natural disasters, drought is one of the most severe threats to all living beings globally. Due to global climate change, the frequency and duration of droughts have increased in many parts of the world. Therefore, accurate prediction and forecasting of droughts are essential for effective mitigation policies and sustainable research. In recent research, the use of ensemble global climate models (GCMs) for simulating precipitation data is common. The objective of this research is to enhance the multi-model ensemble (MME) for improving future drought characterizations. In this research, we propose the use of relative importance metric (RIM) to address collinearity effects and point-wise discrepancy weights (PWDW) in GCMs. Consequently, this paper introduces a new statistical framework for weighted ensembles called the discrepancy-enhanced beta weighting ensemble (DEBWE). DEBWE enhances the weighted ensemble data of precipitation simulated by multiple GCMs. In DEBWE, we addressed uncertainties in GCMs arising from collinearity and outliers. To evaluate the effectiveness of the proposed weighting framework, we compared its performance with the simple average multi-model ensemble (SAMME), Taylor skill score ensemble (TSSE), and mutual information ensemble (MIE). Based on the Kling-Gupta efficiency (KGE) metric, DEBWE outperforms all competitors across all evaluation criteria. These inferences are based on the analysis of historical simulated data from 22 GCMs in the CMIP6 project. The quantitative performance indicators strongly support the superiority of DEBWE. The median and mean KGE values for DEBWE are 0.2650 and 0.2429, compared to SAMME (0.1000, 0.0991), TSSE (0.2600, 0.2397), and MIE (0.1550, 0.1511). For drought assessment, we computed the adaptive standardized precipitation index (SPI) for three future scenarios: SSP1-2.6, SSP2-4.5, and SSP5-8.5. The steady-state probabilities suggest that normal drought (ND) is the most frequent condition, with extreme events (dry or wet) being less probable.

Wasserstein Auto-Encoders of Merge Trees (and Persistence Diagrams).

This article presents a computational framework for the Wasserstein auto-encoding of merge trees (MT-WAE), a novel extension of the classical auto-encoder neural network architecture to the Wasserstein metric space of merge trees. In contrast to traditional auto-encoders which operate on vectorized data, our formulation explicitly manipulates merge trees on their associated metric space at each layer of the network, resulting in superior accuracy and interpretability. Our novel neural network approach can be interpreted as a non-linear generalization of previous linear attempts (Pont et al. 2023) at merge tree encoding. It also trivially extends to persistence diagrams. Extensive experiments on public ensembles demonstrate the efficiency of our algorithms, with MT-WAE computations in the orders of minutes on average. We show the utility of our contributions in two applications adapted from previous work on merge tree encoding (Pont et al. 2023). First, we apply MT-WAE to merge tree compression, by concisely representing them with their coordinates in the final layer of our auto-encoder. Second, we document an application to dimensionality reduction, by exploiting the latent space of our auto-encoder, for the visual analysis of ensemble data. We illustrate the versatility of our framework by introducing two penalty terms, to help preserve in the latent space both the Wasserstein distances between merge trees, as well as their clusters. In both applications, quantitative experiments assess the relevance of our framework. Finally, we provide a C++ implementation that can be used for reproducibility.

Development and Validation of NOAA’s 20-year global wave ensemble reforecast

Abstract A new 20-year wave reforecast was generated based on the NOAA Global Ensemble Forecast System, version 12 (GEFSv12). It was produced using the same wave model setup as the NCEP’s operational GEFSv12 wave component, which employs the numerical wave model WAVEWATCH III and utilizes three grids with spatial resolutions of 0.2° and 0.25°. The reforecast comprises 5 members with one cycle per day and forecast range of 16 days. Once a week, it expands to 35 days and 11 members. This paper describes the development of the wave ensemble reforecast, focusing primarily on validation against buoys and altimeters. The statistical analyses demonstrated very good performance in the short range for significant wave height, with correlation coefficients of 0.95-0.96 on day 1, and between 0.86-0.88 within week 1, along with bias close to zero. After day 10, correlation coefficients fall below 0.70. We found that the degradation of predictability and the increase in scatter errors predominantly occur in the forecast lead time between days 4 and 10, in terms of the ensemble mean and individual members, including the control. For week 2 and beyond, a probabilistic spatio-temporal analysis of the ensemble space provides useful forecast guidance. Our results provide a framework for expanding the usefulness of wave ensemble data in operational forecasting applications.

Improved snow depth estimation on the Tibetan Plateau using AMSR2 and ensemble learning models

Snow depth (SD) is essential for studying climate change and hydrological cycle on the Tibetan Plateau (TP). Despite the effectiveness of passive microwave remote sensing for large-scale SD measurement, its low spatial resolution and scanning gaps limit its application, particularly in the TP region where the terrain is complex and snow distribution exhibits obvious heterogeneity. This study developed Advanced Microwave Scanning Radiometer 2 (AMSR2) SD downscaling models for the TP using ensemble learning methods and AMSR2 brightness temperature data from October 1, 2012, to April 30, 2021. We employed five ensemble methods—AdaBoost, GBDT, XGBoost, LightGBM, and Random Forest—with LightGBM achieving the highest accuracy (RMSE=2.66 cm). Recursive feature elimination (RFE) was applied to the LightGBM model, optimizing factor selection and maintaining high accuracy. The models excelled in estimating shallow snow areas (SD<5 cm) with an RMSE of 1.60 cm. SHapley Additive exPlanations (SHAP) values were used to quantify global and local contributions of each factor in the modeling process. Key factors included snow cover days, meteorological influences, and brightness temperature (BT) at 89 GHz with horizontal polarization, although their contributions varied significantly across the TP due to environmental gradients. The resulting 500 m SD estimates offer detailed and accurate snow distribution information in complex mountainous regions. Our results help to improve water resource management and climate change analysis on the TP.

Changing dynamics of Western European summertime cut‐off lows: A case study of the July 2021 flood event

AbstractIn July 2021, a cut‐off low‐pressure system brought extreme precipitation to Western Europe. Record daily rainfall totals led to flooding that caused loss of life and substantial damage to infrastructure. Climate change can amplify rainfall extremes via thermodynamic processes, but the role of dynamical changes is uncertain. We assess how the dynamics involved in this particular event are changing using flow analogues. Using past and present periods in reanalyses and large ensemble climate model data of the present‐day climate and 2°C warmer climate, we find that the best flow analogues become more similar to the cut‐off low‐pressure system observed over Western Europe in 2021. This may imply that extreme rain events will occur more frequently in the future. Moreover, the magnitude of the analogue lows has deepened, and the associated air masses contain more precipitable water. Simulations of future climate show similar events of the future could lead to intense rainfall further east than in the current climate, due to a shift of the pattern. Such unprecedented events can have large consequences for society, we need to mitigate and adapt to reduce future impacts.

Enhanced regional ocean ensemble data assimilation through atmospheric coupling in the SKRIPS model

We investigate the impact of ocean data assimilation using the Ensemble Adjustment Kalman Filter (EAKF) from the Data Assimilation Research Testbed (DART) on the oceanic and atmospheric states of the Red Sea. Our study extends the ocean data assimilation experiment performed by Sanikommu et al. (2020) by utilizing the SKRIPS model coupling the MITgcm ocean model and the Weather Research and Forecasting (WRF) atmosphere model. Using a 50-member ensemble, we assimilate satellite-derived sea surface temperature and height and in situ temperature and salinity profiles every three days for one year, starting January 01 2011. Atmospheric data are not assimilated in the experiments. To improve the ensemble realism, perturbations are added to the WRF model using several physics options and the stochastic kinetic energy backscatter (SKEB) scheme. Compared with the control experiments using uncoupled MITgcm with ECMWF ensemble forcing, the EAKF ensemble mean oceanic states from the coupled model are better or insignificantly worse (root-mean-square errors are 23% to −1.3% smaller), especially when the atmospheric model uncertainties are accounted for with stochastic perturbations. We hypothesize that the ensemble spreads of the air–sea fluxes are better represented in the downscaled WRF ensembles when uncertainties are well accounted for, leading to improved representation of the ensemble oceanic states from the new experiments with the coupled model. This indicates the ocean model assimilation will be improved with coupled models and may relax the need for operational centers to provide atmospheric ensembles to drive ocean forecasts. Although the feedback from ocean to atmosphere is included in this two-way regional coupled configuration, we find no significant effect of ocean data assimilation on the ensemble mean latent heat flux and 10-m wind speed over the Red Sea. This suggests that the improved skill using the coupled model is not from the two-way coupling, but from downscaling the ensemble atmospheric forcings (one-way coupled) to drive the ocean model.

Using Satellite Data Assimilation Techniques to Combine Infrasound Observations and a Full Ray-Tracing Model to Constrain Stratospheric Variables

Abstract Infrasound waves generated at Earth’s surface can reach high altitudes before returning to the surface to be recorded by microbarometer array stations. These waves carry information about the propagation medium, in particular temperature and winds in the atmosphere. It is only recently that studies on the assimilation of such data into atmospheric models have been published. Intending to advance this line of research, we here use the modulated ensemble transform Kalman filter (METKF)—commonly used in satellite data assimilation—to assimilate infrasound-related observations in order to update a column of three vertically varying variables: temperature and horizontal wind speeds. This includes stratospheric and mesospheric heights, which are otherwise poorly observed. The numerical experiments on synthetic data but with realistic reanalysis product atmospheric specifications (following the observing system simulation experiment paradigm) reveal that a large ensemble is capable of reducing errors, especially for wind speeds in stratospheric heights close to 30–60 km. While using a small ensemble leads to incorrect analysis increments and large estimation errors, the METKF ameliorates this problem and even achieves error reduction from the prior to the posterior mean estimator. Significance Statement The stratosphere and mesosphere have significantly less observational coverage compared to the troposphere, especially for the winds. This lack of information can reduce the accuracy of medium-range weather forecasts. By mimicking a realistic setup, this study paves the way for including novel observations in the estimation of the atmospheric state in these heights using an ensemble data assimilation method. These observations come from a dataset of opportunity containing infrasound-related measurements that are routinely carried out at several stations around the world.

Projecting Climate Change Impact on Precipitation Patterns during Different Growth Stages of Rainfed Wheat Crop in the Pothwar Plateau, Pakistan

In rainfed areas, precipitation variations directly impact wheat growth stages such as emergence, tillering, jointing and booting, and maturity. Evaluating the impact of climate change on precipitation patterns during these critical growth stages is crucial for adapting climate change and ensuring global food security. In this study, projections of five General Circulation models (GCMs) under two shared socioeconomic pathways (SSPs) were used to predict the changing characteristics of precipitation during four main growth stages of wheat in the rainfed region of the Pothwar Plateau, Pakistan. Historical datasets of daily precipitation at six weather stations were analyzed to check the past changes in the precipitation patterns. During the baseline period (1985–2014), the annual average precipitation decreased at a rate of −9.75 mm/decade, while the amount of precipitation during the rabi season (wheat-growing season) decreased at a rate of −20.47 mm/decade. An increase in the precipitation was found during the fourth (flowering) stage of crop growth, while the first three stages experienced a decrease in the precipitation amount. The multimodal ensembled data, under the SSP2-4.5 scenario, revealed a significant decline (at the rate of −16.63 mm/decade) in the future annual precipitation. However, it is projected that, under SSP2-4.5, there may be a slight increase (4.03 mm/decade) in the total precipitation amount during the future rabi season. Under the SSP5-8.5 scenario, average annual precipitation exhibited a slightly increasing trend, increasing by 1.0 mm/decade. However, during the rabi season, there was a possibility of a decrease in precipitation amount, with a rate of 11.64 mm/decade. It is also expected that the precipitation amount may vary significantly during the crown root initiation, jointing and booting, and flowering stages in the near future. These results provide a framework for the planning of wheat production in the Pothwar region of Pakistan, taking into account the potential impact of shifting weather patterns, particularly in terms of uneven precipitation.

Multidecadal Changes of the Seasonal Potential Predictability of Winter PNA and Associated Circulation Anomalies

Abstract Utilizing ensemble hindcast data from the Community Earth System Model (CESM) spanning the years 1900–2014, the multidecadal changes in the seasonal potential predictability of the winter Pacific–North American (PNA) teleconnection pattern and associated circulation anomalies have been investigated by using an information-based metric of relative entropy and the method of the most predictable component analysis. Results show that the seasonal potential predictability of winter PNA has significant multidecadal changes, with values much higher at the two ends of the twentieth century and much lower in between particularly in the 1930s and 1940s. The changes in the seasonal potential predictability of winter PNA are mostly reflected by the temporal evolutions of PNA rather than the location changes of active centers. Further, the changes are mostly contributed by the external forcing of El Niño–Southern Oscillation (ENSO)-related sea surface temperature anomalies in tropical central and eastern Pacific. In particular, the combined effects of lower amplitudes, reduced persistence, and a more eastward shift in warming centers lead to the reduced seasonal potential predictability of PNA and associated circulation changes in the 1930s and 1940s. Significance Statement Seasonal prediction of the winter Pacific–North American (PNA) teleconnection pattern and associated circulation anomalies is very important due to its profound climate impacts. Understanding the multidecadal fluctuations and its driving sources of the potential predictability of winter PNA and associated circulation anomalies are meaningful for skillful seasonal prediction of winter PNA and circulation anomalies as well as related climate variations. This study for the first time shows that the multidecadal fluctuations of the potential predictability of winter PNA are quite significant and the changes are mostly reflected by its temporal evolutions rather than spatial shifts of active centers. Furthermore, this study shows that the strength, persistence, and warming center locations of ENSO-related sea surface temperatures in tropical Pacific play a crucial role on the multidecadal changes of the potential predictability of winter PNA and associated circulation anomalies.

Prediction of material toughness using ensemble learning and data augmentation

ABSTRACT The present work investigates the impact resistance of metallic parts produced using Laser Powder Bed Fusion and the possibility of its prediction using machine learning algorithms. The challenge lies in finding optimal process parameters before printing based on the existing data. Economic constraints often result in the availability of only a limited amount of data for predictive purposes. In this work, around one hundred data points from Charpy impact tests on AlSi10Mg0.5 were used to analyse the correlation between the impact resistance and process parameters, including information about sample porosity. The present research implements a data augmentation technique that artificially increases the volume of training data by applying domain-specific transformations to the original limited dataset. Using this technique, the dataset had been extended to over one thousand data points. To identify the most suitable approach for the specific issue at hand, several algorithms were explored: Regression Neural Network, K-Nearest Neighbours, Decision Tree, Random Forest, AdaBoost, Gradient Boosting, XGBoost, as well as ensemble combinations of Random Forest with AdaBoost, Gradient Boosting, and XGBoost algorithms. The results suggest that the Random Forest and the boosting algorithms generalise best given the sparse testing data. The best-performing models yield a prediction fitness reaching 86 percent. Therefore, an effective model for predicting the impact resistance had been developed and can be used to optimise the quality of additively manufactured parts.

An Ensemble Machine Learning and Data Mining Approach to Enhance Stroke Prediction.

Stroke poses a significant health threat, affecting millions annually. Early and precise prediction is crucial to providing effective preventive healthcare interventions. This study applied an ensemble machine learning and data mining approach to enhance the effectiveness of stroke prediction. By employing the cross-industry standard process for data mining (CRISP-DM) methodology, various techniques, including random forest, ExtraTrees, XGBoost, artificial neural network (ANN), and genetic algorithm with ANN (GANN) were applied on two benchmark datasets to predict stroke based on several parameters, such as gender, age, various diseases, smoking status, BMI, HighCol, physical activity, hypertension, heart disease, lifestyle, and others. Due to dataset imbalance, Synthetic Minority Oversampling Technique (SMOTE) was applied to the datasets. Hyperparameter tuning optimized the models via grid search and randomized search cross-validation. The evaluation metrics included accuracy, precision, recall, F1-score, and area under the curve (AUC). The experimental results show that the ensemble ExtraTrees classifier achieved the highest accuracy (98.24%) and AUC (98.24%). Random forest also performed well, achieving 98.03% in both accuracy and AUC. Comparisons with state-of-the-art stroke prediction methods revealed that the proposed approach demonstrates superior performance, indicating its potential as a promising method for stroke prediction and offering substantial benefits to healthcare.

Improving ensemble data assimilation through Probit-space Ensemble Size Expansion for Gaussian Copulas (PESE-GC)

Abstract. Small forecast ensemble sizes (&lt; 100) are common in the ensemble data assimilation (EnsDA) component of geophysical forecast systems, thus limiting the error-constraining power of EnsDA. This study proposes an efficient and embarrassingly parallel method to generate additional ensemble members: the Probit-space Ensemble Size Expansion for Gaussian Copulas (PESE-GC; “peace gee see”). Such members are called “virtual members”. PESE-GC utilizes the users' knowledge of the marginal distributions of forecast model variables. Virtual members can be generated from any (potentially non-Gaussian) multivariate forecast distribution that has a Gaussian copula. PESE-GC's impact on EnsDA is evaluated using the 40-variable Lorenz 1996 model, several EnsDA algorithms, several observation operators, a range of EnsDA cycling intervals, and a range of forecast ensemble sizes. Significant improvements to EnsDA (p&lt;0.01) are observed when either (1) the forecast ensemble size is small (≤20 members), (2) the user selects marginal distributions that improve the forecast model variable statistics, and/or (3) the rank histogram filter is used with non-parametric priors in high-forecast-spread situations. These results motivate development and testing of PESE-GC for EnsDA with high-order geophysical models.

Developing an Explainable Variational Autoencoder (VAE) Framework for Accurate Representation of Local Circulation in Taiwan

AbstractThis study develops an explainable variational autoencoder (VAE) framework to efficiently generate high‐fidelity local circulation patterns in Taiwan, ensuring an accurate representation of the physical relationship between generated local circulation and upstream synoptic flow regimes. Large ensemble semi‐realistic simulations were conducted using a high‐resolution (2 km) model, TaiwanVVM, where critical characteristics of various synoptic flow regimes were carefully selected to focus on the effects of local circulation variations. The VAE was constructed to capture essential representations of local circulation scenarios associated with the lee vortices by training on the ensemble data set. The VAE's latent space effectively captures the synoptic flow regimes as controlling factors, aligning with the physical understanding of Taiwan's local circulation dynamics. The critical transition of flow regimes under the influence of southeasterly synoptic flow regimes is also well represented in the VAE's latent space. This indicates that the VAE can learn the nonlinear characteristics of the multiscale interactions involving the lee vortex. The latent space within VAE can serve as a reduced‐order model for predicting local circulation using synoptic wind speed and direction. This explainable VAE binds the physical reasoning to the predictions of the local circulation that ensures the physical examination of the uncertainty in accelerating the local weather assessments under various climate change scenarios.

Building Materials Classification Model Based on Text Data Enhancement and Semantic Feature Extraction

In order to accurately extract and match carbon emission factors from the Chinese textual building materials list and construct a precise carbon emission factor database, it is crucial to accurately classify the textual building materials. In this study, a novel classification model based on text data enhancement and semantic feature extraction is proposed and applied for building materials classification. Firstly, the explanatory information on the building materials is collected and normalized to construct the original dataset. Then, the Latent Dirichlet Allocation and statistical-language-model-based hybrid ensemble data enhancement methods are explained in detail, and the semantic features closely related to the carbon emission factor are extracted by constructed composite convolutional networks and the transformed word vectors. Finally, the ensemble classification model is designed, constructed, and applied to match the carbon emission factor from the textual building materials. The experimental results show that the proposed model improves the F1Macro score by 4–12% compared to traditional machine learning and deep learning models.

Sample-efficient reinforcement learning with knowledge-embedded hybrid model for optimal control of mining industry

The froth flotation process is a cost-effective and widely employed method for mineral separation. A core challenge is to maximize mineral recovery while maintaining a specified minimum concentrate grade. This requires precise control of aeration and slurry level and overcoming disturbances. The rapid advancements in reinforcement learning present a promising approach for controlling froth flotation. However, reinforcement learning suffers from sample inefficiency, which significantly hampers its application to real-world problems. In this research, we proposed an innovative sample-efficient model-based reinforcement learning algorithm to enhance flotation performance by directly leveraging the dynamics of the slurry phase. Our proposed algorithm involves the construction of a hybrid model, comprising a physical model and a data residual model, which effectively learns the underlying dynamics of the flotation process during the interaction between the reinforcement learning agent and process. The physical model integrates domain knowledge into the hybrid model to expedite the training process and enhance the interpretability and generalizability of the hybrid model. To capture uncertainty and address unmodeled aspects of the physical model, we employ an ensemble data residual model. Actor and critic are augmented with fuzzy representation modules based on fuzzy logic inference to improve the learning capacity of the networks and the multi-head critic is applied to reduce overestimation biases and enhance training stability. Case studies demonstrate that, compared to the baseline algorithm, our proposed algorithm requires merely 86.8% of the samples employed by conventional model-based reinforcement learning algorithm and improves the long-term return by at least 12.0%, underscoring the role of domain knowledge in facilitating efficient reinforcement learning.

WRF-PDAF v1.0: implementation and application of an online localized ensemble data assimilation framework

Abstract. Data assimilation is a common technique employed to estimate the state and its associated uncertainties in numerical models. Ensemble-based methods are a prevalent choice, although they can be computationally expensive due to the required ensemble integrations. In this study, we enhance the capabilities of the Weather Research and Forecasting–Advanced Research WRF (WRF-ARW) model by coupling it with the Parallel Data Assimilation Framework (PDAF) in a fully online mode. Through minimal modifications to the WRF-ARW model code, we have developed an efficient data assimilation system. This system leverages parallelization and in-memory data transfers between the model and data assimilation processes, greatly reducing the need for file I/O and model restarts during assimilation. We detail the necessary program modifications in this study. One advantage of the resulting assimilation system is a clear separation of concerns between data assimilation method development and model application resulting from PDAF's model-agnostic structure. To evaluate the assimilation system, we conduct a twin experiment simulating an idealized tropical cyclone. Cycled data assimilation experiments focus on the impact of temperature profiles. The assimilation not only significantly enhances temperature field accuracy but also improves the initial U and V fields. The assimilation process introduces only minimal overhead in runtime when compared to the model without data assimilation and exhibits excellent parallel performance. Consequently, the online WRF-PDAF system emerges as an efficient framework for implementing high-resolution mesoscale forecasting and reanalysis.

A Quantile-Conserving Ensemble Filter Framework. Part III: Data Assimilation for Mixed Distributions with Application to a Low-Order Tracer Advection Model

Abstract The uncertainty associated with many observed and modeled quantities of interest in Earth system prediction can be represented by mixed probability distributions that are neither discrete nor continuous. For instance, a forecast probability of precipitation can have a finite probability of zero precipitation, consistent with a discrete distribution. However, nonzero values are not discrete and are represented by a continuous distribution; the same is true for rainfall rate. Other examples include snow depth, sea ice concentration, amount of a tracer or the source rate of a tracer. Some Earth system model parameters may also have discrete or mixed distributions. Most ensemble data assimilation methods do not explicitly consider the possibility of mixed distributions. The Quantile Conserving Ensemble Filtering Framework (Anderson 2022, 2023) is extended to explicitly deal with discrete or mixed distributions. An example is given using bounded normal rank histogram probability distributions applied to observing system simulation experiments in a low-order tracer advection model. Analyses of tracer concentration and tracer source are shown to be improved when using the extended methods. A key feature of the resulting ensembles is that there can be ensemble members with duplicate values. An extension of the rank histogram diagnostic method to deal with potential duplicates shows that the ensemble distributions from the extended assimilation methods are more consistent with the truth.

Tailoring data assimilation to discontinuous Galerkin models

AbstractIn recent years discontinuous Galerkin (DG) methods have received increased interest from the geophysical community. In these methods the solution in each grid cell is approximated as a linear combination of basis functions. Ensemble data assimilation (DA) aims to approximate the true state by combining model outputs with observations using error statistics estimated from an ensemble of model runs. Ensemble data assimilation in geophysical models faces several well‐documented issues. In this work we exploit the expansion of the solution in DG basis functions to address some of these issues. Specifically, it is investigated whether a DA–DG combination (a) mitigates the need for observation thinning, (b) reduces errors in the field's gradients, and (c) can be used to set up scale‐dependent localisation. Numerical experiments are carried out using stochastically generated ensembles of model states, with different noise properties, and with Legendre polynomials as basis functions. It is found that strong reduction in the analysis error is achieved by using DA–DG and that the benefit increases with increasing DG order. This is especially the case when small scales dominate the background error. The DA improvement in the first derivative is, on the other hand, marginal. We think this to be a counter‐effect of the power of DG to fit the observations closely, which can deteriorate the estimates of the derivatives. Applying optimal localisation to the different polynomial orders, thus exploiting their different spatial length, is beneficial: it results in a covariance matrix closer to the true covariance than the matrix obtained using traditional optimal localisation in state space.