Sequential Data Assimilation Techniques Research Articles

Assimilation of wheat and soil states for improved yield prediction: The APSIM-EnKF framework

Context.Accurate prediction of within-field crop yield in response to spatial and temporal variability provides essential information for farm managers to improve productivity and ensure optimal use of inputs. Understanding yield spatial and temporal variability cannot be solely addressed by crop modelling or remote sensing but by integrating the instantaneous spatial information from remote sensing and the temporal information from crop modelling. Sequential data assimilation techniques allow wheat and soil observations to be assimilated into the crop model while it evolves and evaluate model and observational uncertainties to improve the accuracy of crop monitoring and yield prediction. ObjectivesThe objective of this study was to comprehensively explore the potential yield estimation improvement by assimilating observations of all prognostic wheat and soil states, including various repeat intervals and accuracy, allowing recommendations on implementation to be made. MethodsThis study develops an Ensemble Kalman filter (EnKF) data assimilation framework for the APSIM-Wheat model and illustrates potential improvements in wheat yield estimation through a synthetic study. Through several scenarios, assimilation of wheat and soil observations into APSIM was explored, by assimilating these variables solely or collectively, and in various phenological stages.Results and conclusions.The results showed, under the specific weather and soil conditions assumed in this study, that while open-loop (no data assimilation) provided a yield estimation with a bias of 10.1%, assimilation in the flowering to end of grain filling stage reduced the bias to 1.4%, 2.9%, 4.4%, and 1.0% when constraining with leaf area index, leaf weight, stem weight, surface soil moisture observations, respectively. When assimilating in the floral initiation to the flowering stage, the yield estimation bias was reduced to 7.1%, 9.8%, 1.1%, and 1.2% when constraining with leaf nitrogen, stem nitrogen, top-layer soil ammonium‑nitrogen and nitrate‑nitrogen, respectively. Leaf area index, biomass and surface soil moisture are recommended for data assimilation especially with observations from remote sensing. SignificanceThis study developed a data assimilation framework for the APSIM-Wheat model and can be extended to over 20 crop modules integrated with APSIM. This synthetic study provided a exhaustive data assimilation experiment for wheat and soil states that are measurable by current techniques with a rigorous justification on uncertainties. It thus provides a guide for future agricultral data assimilation practices in choosing crop and soil states for assimilation, and for planning the timing and frequency of data collection. It should also inspire researchers to develop new techniques for measuring wheat states.

Indoor Temperature Forecast based on the Lattice Boltzmann method and Data Assimilation

Control of the indoor environment in buildings is reactive and usually based on sensor data at a single location in a room without any consideration of the air flows within the space. The ability to accurately simulate indoor conditions would enable predictive control offering improved environmental conditions and better energy efficiency. This study investigates the temporal prediction of temperature and its coupling with sensor data for improving thermal comfort forecasting in an indoor environment. We present a real-time implementation of a computational fluid dynamics (CFD) Lattice Boltzmann method (LBM) model coupled with data assimilation (DA) to make periodic updates to the state of the model. Variational and sequential DA techniques are evaluated and two novel methods for real-time accurate temperature prediction are presented. The models are demonstrated for prediction of temperature in an idealized room, with simulated temperature sensor readings used to update LBM-based flow predictions. The LBM-DA approach overcomes the need for accurate boundary conditions while considerably improving the transient and spatial prediction of temperature in terms of root mean squares error, thereby avoiding large deviation from the room’s true flow state. The accuracy of prediction is shown to depend on number of sensors with poor prediction using just a single sensor. Results also depend on quality of sensor data, with highly variable data yielding poorer results. The approach has potential for application in indoor environments to provide more accurate and faster response of control systems to changing environmental conditions.

Sequential data assimilation for real-time probabilistic flood inundation mapping

Abstract. Real-time probabilistic flood inundation mapping is crucial for flood risk warning and decision-making during the emergency period before an upcoming flood event. Considering the high uncertainties involved in the modeling of a nonlinear and complex flood event, providing a deterministic flood inundation map can be erroneous and misleading for reliable and timely decision-making. The conventional flood hazard maps provided for different return periods cannot also represent the actual dynamics of flooding rivers. Therefore, a real-time modeling framework that forecasts the inundation areas before the onset of an upcoming flood is of paramount importance. Sequential data assimilation (DA) techniques are well known for real-time operation of physical models while accounting for existing uncertainties. In this study, we present a DA hydrodynamic modeling framework where multiple gauge observations are integrated into the LISFLOOD-FP model to improve its performance. This study utilizes the ensemble Kalman filter (EnKF) in a multivariate fashion for dual estimation of model state variables and parameters where the correlations among point source observations are taken into account. First, a synthetic experiment is designed to assess the performance of the proposed approach; then the method is used to simulate the Hurricane Harvey flood in 2017. Our results indicate that the multivariate assimilation of point source observations into hydrodynamic models can improve the accuracy and reliability of probabilistic flood inundation mapping by 5 %–7 %, while it also provides the basis for sequential updating and real-time flood inundation mapping.

Data assimilation: The Schrödinger perspective

Data assimilation addresses the general problem of how to combine model-based predictions with partial and noisy observations of the process in an optimal manner. This survey focuses on sequential data assimilation techniques using probabilistic particle-based algorithms. In addition to surveying recent developments for discrete- and continuous-time data assimilation, both in terms of mathematical foundations and algorithmic implementations, we also provide a unifying framework from the perspective of coupling of measures, and Schrödinger’s boundary value problem for stochastic processes in particular.

Evaluation of a Data Assimilation System for Land Surface Models Using CLM4.5

AbstractThe magnitude and persistence of land carbon (C) pools influence long‐term climate feedbacks. Interactive ecological processes influence land C pools and our understanding of these processes is imperfect so land surface models have errors and biases when compared to each other and to real observations. Here we implement an Ensemble Adjustment Kalman Filter (EAKF), a sequential state data assimilation technique to reduce these errors and biases. We implement the EAKF using the Data Assimilation Research Testbed coupled with the Community Land Model (CLM 4.5 in CESM 1.2). We assimilated simulated and real satellite observations for a site in central New Mexico, United States. A series of observing system simulation experiments allowed assessment of the data assimilation system without model error. This showed that assimilating biomass and leaf area index observations decreased model error in C dynamics forecasts (29% using biomass observations and 40% using leaf area index observations) and that assimilation in combination shows greater improvement (51% using both observation streams). Assimilating real observations highlighted likely model structural errors and we implemented an adaptive model‐variance‐inflation technique to allow the model to track the observations. Monthly and longer model forecasts using real observations were improved relative to forecasts without data assimilation. The reliable forecast lead‐time varied by model pool and is dependent on how tightly the C pool is coupled to meteorologically driven processes. The EAKF and similar state data assimilation techniques could reduce errors in projections of the land C sink and provide more robust forecasts of C pools and land‐atmosphere exchanges.

Possible deep connection between volcanic systems evidenced by sequential assimilation of geodetic data

The existence of possible deep connections between nearby volcanoes has so far only been formulated on the basis of correlation in their eruptive activities or geochemical arguments. The use of geodetic data to monitor the deep dynamics of magmatic systems and the possible interference between them has remained limited due to the lack of techniques to follow transient processes. Here, for the first time, we use sequential data assimilation technique (Ensemble Kalman Filter) on ground displacement data to evaluate a possible interplay between the activities of Grímsvötn and Bárðarbunga volcanoes in Iceland. Using a two-reservoir dynamical model for the Grímsvötn plumbing system and assuming a fixed geometry and constant magma properties, we retrieve the temporal evolution of the basal magma inflow beneath Grímsvötn that drops by up to 85% during the 10 months preceding the initiation of the Bárðarbunga rifting event. We interpret the loss of at least 0.016 km3 in the magma supply of Grímsvötn as a consequence of magma accumulation beneath Bárðarbunga and subsequent feeding of the Holuhraun eruption 41 km away. We demonstrate that, in addition to its interest for predicting volcanic eruptions, sequential assimilation of geodetic data has a unique potential to give insights into volcanic system roots.

Assessing sequential data assimilation techniques for integrating GRACE data into a hydrological model

The time-variable terrestrial water storage (TWS) products from the Gravity Recovery And Climate Experiment (GRACE) have been increasingly used in recent years to improve the simulation of hydrological models by applying data assimilation techniques. In this study, for the first time, we assess the performance of the most popular data assimilation sequential techniques for integrating GRACE TWS into the World-Wide Water Resources Assessment (W3RA) model. We implement and test stochastic and deterministic ensemble-based Kalman filters (EnKF), as well as Particle filters (PF) using two different resampling approaches of Multinomial Resampling and Systematic Resampling. These choices provide various opportunities for weighting observations and model simulations during the assimilation and also accounting for error distributions. Particularly, the deterministic EnKF is tested to avoid perturbing observations before assimilation (that is the case in an ordinary EnKF). Gaussian-based random updates in the EnKF approaches likely do not fully represent the statistical properties of the model simulations and TWS observations. Therefore, the fully non-Gaussian PF is also applied to estimate more realistic updates. Monthly GRACE TWS are assimilated into W3RA covering the entire Australia. To evaluate the filters performances and analyze their impact on model simulations, their estimates are validated by independent in-situ measurements. Our results indicate that all implemented filters improve the estimation of water storage simulations of W3RA. The best results are obtained using two versions of deterministic EnKF, i.e. the Square Root Analysis (SQRA) scheme and the Ensemble Square Root Filter (EnSRF), respectively, improving the model groundwater estimations errors by 34% and 31% compared to a model run without assimilation. Applying the PF along with Systematic Resampling successfully decreases the model estimation error by 23%.

Application of Sequential Data-Assimilation Techniques in Groundwater Contaminant Transport Modeling

AbstractGroundwater contaminant transport modeling is basically performed to predict contaminant concentration and to understand the biochemical and physical processes that happen in the subsurface of porous media. Modelers have been faced with the challenge of accurately modeling the behavior and fate of contaminants in groundwater with models and techniques that incorporate the appropriate noise statistics and estimates the hydrogeologic parameters effectively. Unaccounted noise and uncertainties in the modeling greatly affect the accuracy of these predictions. In this paper, two Monte Carlo-based techniques, particle filter (PF) and Ensemble Kalman filter (EnKF), were applied to a three-dimensional (3D) groundwater contaminant transport model to accurately estimate the first-order decay rate and contaminant concentration at each time step. The PF and EnKF are embedded with Sampling Importance Resampling (SIR) and Singular Value Decomposition (SVD) concepts to avoid degeneracy and matrix singularity, re...

Particle MCMC for Bayesian Microwave Control

We consider the problem of local radioelectric property estimation from global electromagnetic scattering measurements. This challenging ill-posed high dimensional inverse problem can be explored by intensive computations of a parallel Maxwell solver on a petaflopic supercomputer. Then, it is shown how Bayesian inference can be perfomed with a Particle Marginal Metropolis-Hastings (PMMH) approach, which includes a Rao-Blackwellised Sequential Monte Carlo algorithm with interacting Kalman filters. Material properties, including a multiple components "Debye relaxation"/"Lorenzian resonant" material model, are estimated; it is illustrated on synthetic data. Eventually, we propose different ways to deal with higher dimensional problems, from parallelization to the original introduction of efficient sequential data assimilation techniques, widely used in weather forecasting, oceanography, geophysics, etc.

Bayesian inference applied to spatio-temporal reconstruction of flows around a NACA0012 airfoil

In this paper, we shall investigate sequential data assimilation techniques to improve the stability of reduced-order models for fluid flows. The reduced-order model used relies on a Galerkin projection of Navier–Stokes equations on proper orthogonal decomposition (POD) basis vectors estimated from snapshots of the flow fields obtained with time-resolved particle image velocimetry (TR-PIV) measurements. The coefficients of the dynamical system are given through a least-squares regression technique applied to the experimental data and lead to a low-order model which is known to diverge, or damp, rapidly in time if left uncontrolled. In this context, a sequential data assimilation method based on a Bayesian approach is proposed. In this formalism, reduced-order models (ROMs) are modeled with discrete time from the hidden Markov processes. Given the whole trajectories of the POD temporal modes, the state of ROM coefficients initially provided by noisy PIV measurements are re-estimated from a Kalman filtering of the sequential data. Results are obtained for the flow around a NACA0012 airfoil at Reynolds numbers of 1000 and 2000 and angles of attack of \(10^{\circ },15^{\circ },20^{\circ }\) and \(30^{\circ }\).

P14 データ同化による数値計算技術の展開(ALSSポスターセッション/アピールタイム(2))

Data assimilation is a generic technique which combines numerical simulation results with experimental data. In this paper, the ensemble Kalman filter (EnKF), which is categorized as sequential data assimilation techniques, is investigated when applied in fmite element analysis. The EnKF is a nonlinear extension of the standard Kalman filter, where a priori state estimates are provided by using the Monte Carlo method of nonlinear physical simulations. The estimates obtained in the EnKF can inherit many of the nonlinear properties of a priori state estimates. Through a simple numerical experiment, the effectiveness of the EnKF is examined.

REMOTE SENSING AND DATA ASSIMILATION FOR SURF ZONE BATHYMETRIC INVERSION

We demonstrate the implementation and validation of a surf zone forecasting system, which uses remote sensing observations to control errors in surf zone bathymetry. This system uses ensemble-based sequential data assimilation techniques, which are adaptable to arbitrary geophysical observations, and/or arbitrary improvements to model physics. The system is validated using data from a 2010 field experiment at Duck, NC (U.S.A.), and is shown to produce accurate corrections to bathymetry, leading to improvements in prediction of currents.

Data assimilation using the particle filter for identifying the elasto‐plastic material properties of geomaterials

SUMMARYA computational method, incorporating the finite element model (FEM) into data assimilation using the particle filter, is presented for identifying elasto‐plastic material properties based on sequential measurements under the known changing traction boundary conditions to overcome some difficulties in identifying the parameters for elasto‐plastic problems from which the existing inverse analysis strategies have suffered. A soil–water coupled problem, which uses the elasto‐plastic constitutive model, is dealt with as the geotechnical application. Measured data on the settlement and the pore pressure are obtained from a synthetic FEM computation as the forward problem under the known parameters to be identified for both the element tests and the ground behavior during the embankment construction sequence. Parameter identification for elasto‐plastic problems, such as soil behavior, should be made by considering the measurements of deformation and/or pore pressure step by step from the initial stage of construction and throughout the deformation history under the changing traction boundary conditions because of the embankment or the excavation because the ground behavior is highly dependent on the loading history. Thus, it appears that sequential data assimilation techniques, such as the particle filter, are the preferable tools that can provide estimates of the state variables, that is, deformation, pore pressure, and unknown parameters, for the constitutive model in geotechnical practice. The present paper discusses the priority of the particle filter in its application to initial/boundary value problems for elasto‐plastic materials and demonstrates a couple of numerical examples. Copyright © 2012 John Wiley & Sons, Ltd.

Sequential variations of phytoplankton growth and mortality in an NPZ model: A remote-sensing-based assessment

Radiometric data in the visible domain acquired by satellite remote sensing have proven to be powerful for monitoring the states of the ocean, both physical and biological. With the help of these data it is possible to understand certain variations in biological responses of marine phytoplankton on ecological time scales. Here, we implement a sequential data-assimilation technique to estimate from a conventional nutrient–phytoplankton–zooplankton (NPZ) model the time variations of observed and unobserved variables. In addition, we estimate the time evolution of two biological parameters, namely, the specific growth rate and specific mortality of phytoplankton. Our study demonstrates that: (i) the series of time-varying estimates of specific growth rate obtained by sequential data assimilation improves the fitting of the NPZ model to the satellite-derived time series: the model trajectories are closer to the observations than those obtained by implementing static values of the parameter; (ii) the estimates of unobserved variables, i.e., nutrient and zooplankton, obtained from an NPZ model by implementation of a pre-defined parameter evolution can be different from those obtained on applying the sequences of parameters estimated by assimilation; and (iii) the maximum estimated specific growth rate of phytoplankton in the study area is more sensitive to the sea-surface temperature than would be predicted by temperature-dependent functions reported previously. The overall results of the study are potentially useful for enhancing our understanding of the biological response of phytoplankton in a changing environment.

From the Ensemble Kalman Filter to the Particle Filter: a comparative study in rainfall-runoff models

AbstractRainfall-runoff models are hydrologic models and they play a very important role in flood forecasting. Hydrologic systems are featured by a non-linear behaviour, non-Gaussian distributions and the presence of large uncertainties in the model itself and in the input data. Therefore, the use of non-linear/non-Gaussian state estimation or data assimilation techniques is important in order to improve the model predictions. The objective of this paper is to present a comparative study between the performances of two sequential data assimilation techniques, which are based on Monte Carlo (MC) methods. The advantages and drawbacks of the well known ensemble Kalman Filter (EnKF) and the standard particle filter will be discussed in this study. Results show that both filters perform similar with the surface soil moisture as the estimated variable and using synthetic generated data.

Joint Updating of Petrophysical Properties and Discrete Facies Variables From Assimilating Production Data Using the EnKF

Summary The ensemble Kalman filter (EnKF), is a sequential data-assimilation technique that has been shown to work quite well in obtaining conditional facies models from assimilating production data. Because the problem of history matching geological facies is quite complex, most efforts at solving this problem typically assume that facies properties are constant and spatially homogeneous. In this paper, we propose a method for updating both the categorical facies variables and the spatially heterogeneous and nonuniform properties of the facies in a consistent manner within the EnKF framework. Tests of our proposed approach on two representative examples with different features of nonstationarity resulted in satisfactory history-match solutions and geologically consistent estimates of the nonuniform and heterogeneous petrophysical properties.

Data assimilation for distributed hydrological catchment modeling via ensemble Kalman filter

Catchment scale hydrological models are critical decision support tools for water resources management and environment remediation. However, the reliability of hydrological models is inevitably affected by limited measurements and imperfect models. Data assimilation techniques combine complementary information from measurements and models to enhance the model reliability and reduce predictive uncertainties. As a sequential data assimilation technique, the ensemble Kalman filter (EnKF) has been extensively studied in the earth sciences for assimilating in-situ measurements and remote sensing data. Although the EnKF has been demonstrated in land surface data assimilations, there are no systematic studies to investigate its performance in distributed modeling with high dimensional states and parameters. In this paper, we present an assessment on the EnKF with state augmentation for combined state-parameter estimation on the basis of a physical-based hydrological model, Soil and Water Assessment Tool (SWAT). Through synthetic simulation experiments, the capability of the EnKF is demonstrated by assimilating the runoff and other measurements, and its sensitivities are analyzed with respect to the error specification, the initial realization and the ensemble size. It is found that the EnKF provides an efficient approach for obtaining a set of acceptable model parameters and satisfactory runoff, soil water content and evapotranspiration estimations. The EnKF performance could be improved after augmenting with other complementary data, such as soil water content and evapotranspiration from remote sensing retrieval. Sensitivity studies demonstrate the importance of consistent error specification and the potential with small ensemble size in the data assimilation system.

Comparison of Data Assimilation Techniques for a Coupled Model of Surface and Subsurface Flow

Data assimilation in the geophysical sciences refers to methodologies to optimally merge model predictions and observations. The ensemble Kalman filter (EnKF) is a statistical sequential data assimilation technique explicitly developed for nonlinear filtering problems. It is based on a Monte Carlo approach that approximates the conditional probability densities of the variables of interest by a finite number of randomly generated model trajectories. In Newtonian relaxation or nudging (NN), which can be viewed as a special case of the classic Kalman filter, model variables are driven toward observations by adding to the model equations a forcing term, or relaxation component, that is proportional to the difference between simulation and observation. The forcing term contains four‐dimensional weighting functions that can, ideally, incorporate prior knowledge about the characteristic scales of spatial and temporal variability of the state variable(s) being assimilated. In this study, we examined the EnKF and NN algorithms as implemented for a complex hydrologic model that simulates catchment dynamics, coupling a three‐dimensional finite element Richards' equation solver for variably saturated porous media and a finite difference diffusion wave approximation for surface water flow. We report on the retrieval performance of the two assimilation schemes for a small catchment in Belgium. The results of the comparison show that nudging, while more straightforward and less expensive computationally, is not as effective as the ensemble Kalman filter in retrieving the true system state. We discuss some of the strengths and weaknesses, both physical and numerical, of the NN and EnKF schemes.

Assessing parameter, precipitation, and predictive uncertainty in a distributed hydrological model using sequential data assimilation with the particle filter

Summary Sequential data assimilation techniques offer the possibility to handle different sources of uncertainty explicitly in hydrological models and hence improve their predictive capabilities. Amongst the different techniques, sequential Monte Carlo or particle filter methods offer the capability to handle non-linear/non-Gaussian state-space models while preserving the spatial variability of updated state variables, both desirable features when assimilating data in distributed hydrological models. In this work we apply the residual resampling particle filter to assess parameter, precipitation, and predictive uncertainty in the distributed rainfall–runoff model LISFLOOD. First, we compare estimated posterior parameter distributions with results of the Shuffled Complex Evolution Metropolis global optimization algorithm obtained using identical input data for the Meuse catchment and considering parameter uncertainty only. Both approaches result in well identifiable posterior parameter distributions and provide a reasonable fit to the observed hydrograph. The resulting posterior distributions, however, vary considerably in shape, location, and scale, most likely caused by the different assumptions made in the output error model. An evaluation of the predictive distributions illustrates that predictive uncertainty is significantly underestimated for both approaches when accounting for parameter uncertainty only. A second case study shows that considering additionally precipitation uncertainty not only increases the spread of the posterior parameter distributions but may also result in a completely different location and/or shape of the posterior distributions. Evaluation of the posterior precipitation multiplier distribution reveals that no overall systematic bias exists in the precipitation grids and that particle filtering is a suitable tool to quantify and reduce precipitation uncertainty. Furthermore, considering precipitation and parameter uncertainty leads to an improvement in model predictive capabilities, especially for the high flow periods. However, the remaining underestimation of predictive uncertainty also indicates that model structural uncertainty is equally important, in spite of using a physically-based distributed hydrological model that should theoretically provide an improved description of the hydrological system dynamics.

Problems and Techniques

This issues features papers about mathematical genetics, data compression, and data assimilation in oceanography. 1. Computer algebra can help to predict the genetic makeup of a population. This is what we can conclude from the first paper, “Buchberger's Algorithm and the Two-Locus, Two-Allele Model.” With reproduction rates and probability of chromosome recombination as parameters, and one variable for each chromosome type, one can set up a system of polynomial equations. The solution value for a variable indicates how often this chromosome type occurs in the population. The question is now, over all parameter choices, how many different equilibrium solutions can there be? In other words, in the long term, can a chromosome type occur with any possible frequency? Arthur Copeland's answer is no: the number of solutions is almost always finite (and conjectured to be 15) in the special case when a population contains at most four different chromosome types. He arrives at his answer by means of affine varieties, ideals, and Gröbner bases. He also shows that the equilibrium solutions vary smoothly with the parameters. 2. The second paper, “A Direct Formulation for Sparse PCA Using Semidefinite Programming,” by Alexandre d'Aspremont, Laurent El Ghaoui, Michael I. Jordan, and Gert R. G. Lanckriet, is concerned with principal component analysis (PCA), a technique for compressing data so that as little information as possible is lost. PCA accomplishes this by finding directions of maximum variance in the data. In data analysis of gene expressions, for instance, different elements of a direction vector correspond to different genes. However, interpreting the meaning of a direction is much easier if only a few genes contribute to the direction. Such a direction vector has many zero elements and is called sparse. The authors propose a semidefinite programming method that maximizes variance as well as sparsity. The method is computationally efficient and robust. 3. The topic of the third paper, “A Reduced-Order Kalman Filter for Data Assimilation in Physical Oceanography,” by D. Rozier, F. Birol, E. Cosme, P. Brasseur, J. M. Brankart, and J. Verron, is a technique used by oceanographers to increase the accuracy of numerical models for predicting ocean circulation. Data assimilation combines the output of a simulation step with observed data and error statistics and then returns this “corrected” output to the simulation as the basis for the next step. Data assimilation based on Kalman filters assumes that measurement and model errors are unbiased and Gaussian. This is expensive, however, because an error covariance matrix must be constructed at every step. A reduced-order Kalman filter (called SEEK) is a cheaper version that reduces (or compresses) the size of the covariance matrix—the same key idea as in the paper described above. The authors give a very readable account of the issues associated with so-called ocean global circulation models: choice of vertical coordinate system (depending on the ocean region: shallow coastal shelf, steeply sloping ocean floors, stratified regions); limitations of models for ocean circulation simulations; sequential data assimilation techniques; adaptation of Kalman filters to geophysical applications; and modes of data acquisition (via satellites, ships, floats, mooring networks). The effectiveness of the SEEK filter is illustrated with numerical simulations of the Gulf Stream and the Bay of Biscay.