Data Assimilation Problems Research Articles

Learning Variational Data Assimilation Models and Solvers

AbstractData assimilation is a key component of operational systems and scientific studies for the understanding, modeling, forecasting and reconstruction of earth systems informed by observation data. Here, we investigate how physics‐informed deep learning may provide new means to revisit data assimilation problems. We develop a so‐called end‐to‐end learning approach, which explicitly relies on a variational data assimilation formulation. Using automatic differentiation embedded in deep learning framework, the key novelty of the proposed physics‐informed approach is to allow the joint training of the representation of the dynamical process of interest as well as of the solver of the data assimilation problem. We may perform this joint training using both supervised and unsupervised strategies. Our numerical experiments on Lorenz‐63 and Lorenz‐96 systems report significant gain w.r.t. a classic gradient‐based minimization of the variational cost both in terms of reconstruction performance and optimization complexity. Intriguingly, we also show that the variational models issued from the true Lorenz‐63 and Lorenz‐96 ODE representations may not lead to the best reconstruction performance. We believe these results may open new research avenues for the specification of assimilation models for earth systems, both to speed‐up the inversion problem with trainable solvers but possibly more importantly in the way data assimilation systems are designed, for instance regarding the representation of geophysical dynamics.

On the physical nudging equations

In this work we show how it is possible to derive a new set of nudging equations, a tool still used in many data assimilation problems, starting from statistical physics considerations and availing ourselves of stochastic parameterizations that take into account unresolved interactions. The fluctuations used are thought of as Gaussian white noise with zero mean. The derivation is based on the conditioned Langevin dynamics technique. Exploiting the relation between the Fokker–Planck and the Langevin equations, the nudging equations are derived for a maximally observed system that converges towards the observations in finite time. The new nudging term found is the analog of the so called quantum potential of the Bohmian mechanics. In order to make the new nudging equations feasible for practical computations, two approximations are developed and used as bases from which extending this tool to non-perfectly observed systems. By means of a physical framework, in the zero noise limit, all the physical nudging parameters are fixed by the model under study and there is no need to tune other free ad-hoc variables. The limit of zero noise shows that also for the classical nudging equations it is necessary to use dynamical information to correct the typical relaxation term. A comparison of these approximations with a 3DVar scheme, that use a conjugate gradient minimization, is then shown in a series of four twin experiments that exploit low order chaotic models.

A model-independent data assimilation (MIDA) module and its applications in ecology

Abstract. Models are an important tool to predict Earth system dynamics. An accurate prediction of future states of ecosystems depends on not only model structures but also parameterizations. Model parameters can be constrained by data assimilation. However, applications of data assimilation to ecology are restricted by highly technical requirements such as model-dependent coding. To alleviate this technical burden, we developed a model-independent data assimilation (MIDA) module. MIDA works in three steps including data preparation, execution of data assimilation, and visualization. The first step prepares prior ranges of parameter values, a defined number of iterations, and directory paths to access files of observations and models. The execution step calibrates parameter values to best fit the observations and estimates the parameter posterior distributions. The final step automatically visualizes the calibration performance and posterior distributions. MIDA is model independent, and modelers can use MIDA for an accurate and efficient data assimilation in a simple and interactive way without modification of their original models. We applied MIDA to four types of ecological models: the data assimilation linked ecosystem carbon (DALEC) model, a surrogate-based energy exascale earth system model: the land component (ELM), nine phenological models and a stand-alone biome ecological strategy simulator (BiomeE). The applications indicate that MIDA can effectively solve data assimilation problems for different ecological models. Additionally, the easy implementation and model-independent feature of MIDA breaks the technical barrier of applications of data–model fusion in ecology. MIDA facilitates the assimilation of various observations into models for uncertainty reduction in ecological modeling and forecasting.

New bounds on the condition number of the Hessian of the preconditioned variational data assimilation problem

AbstractData assimilation algorithms combine prior and observational information, weighted by their respective uncertainties, to obtain the most likely posterior of a dynamical system. In variational data assimilation the posterior is computed by solving a nonlinear least squares problem. Many numerical weather prediction (NWP) centers use full observation error covariance (OEC) weighting matrices, which can slow convergence of the data assimilation procedure. Previous work revealed the importance of the minimum eigenvalue of the OEC matrix for conditioning and convergence of the unpreconditioned data assimilation problem. In this article we examine the use of correlated OEC matrices in the preconditioned data assimilation problem for the first time. We consider the case where there are more state variables than observations, which is typical for applications with sparse measurements, for example, NWP and remote sensing. We find that similarly to the unpreconditioned problem, the minimum eigenvalue of the OEC matrix appears in new bounds on the condition number of the Hessian of the preconditioned objective function. Numerical experiments reveal that the condition number of the Hessian is minimized when the background and observation lengthscales are equal. This contrasts with the unpreconditioned case, where decreasing the observation error lengthscale always improves conditioning. Conjugate gradient experiments show that in this framework the condition number of the Hessian is a good proxy for convergence. Eigenvalue clustering explains cases where convergence is faster than expected.

A physics perspective on lidar data assimilation for mobile robots

AbstractThis paper presents a new algorithm for lidar data assimilation relying on a new forward model. Current mapping algorithms suffer from multiple shortcomings, which can be related to the lack of clear forward model. In order to address these issues, we provide a mathematical framework where we show how the use of coarse model parameters results in a new data assimilation problem. Understanding this new problem proves essential to derive sound inference algorithms. We introduce a model parameter specifically tailored for lidar data assimilation, which closely relates to the local mean free path. Using this new model parameter, we derive its associated forward model and we provide the resulting mapping algorithm. We further discuss how our proposed algorithm relates to usual occupancy grid mapping. Finally, we present an example with real lidar measurements.

Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs

Abstract Physics-informed neural networks (PINNs) have recently been very successfully applied for efficiently approximating inverse problems for partial differential equations (PDEs). We focus on a particular class of inverse problems, the so-called data assimilation or unique continuation problems, and prove rigorous estimates on the generalization error of PINNs approximating them. An abstract framework is presented and conditional stability estimates for the underlying inverse problem are employed to derive the estimate on the PINN generalization error, providing rigorous justification for the use of PINNs in this context. The abstract framework is illustrated with examples of four prototypical linear PDEs. Numerical experiments, validating the proposed theory, are also presented.

Optimal control problems governed by two dimensional convective Brinkman-Forchheimer equations

<p style='text-indent:20px;'>The convective Brinkman-Forchheimer (CBF) equations describe the motion of incompressible viscous fluids through a rigid, homogeneous, isotropic, porous medium and is given by</p><p style='text-indent:20px;'><disp-formula> <label/> <tex-math id="FE1"> \begin{document}$ \partial_t{\boldsymbol{u}}-\mu \Delta{\boldsymbol{u}}+({\boldsymbol{u}}\cdot\nabla){\boldsymbol{u}}+\alpha{\boldsymbol{u}}+\beta|{\boldsymbol{u}}|^{r-1}{\boldsymbol{u}}+\nabla p = {\boldsymbol{f}},\ \nabla\cdot{\boldsymbol{u}} = 0. $\end{document} </tex-math></disp-formula></p><p style='text-indent:20px;'>In this work, we consider some distributed optimal control problems like total energy minimization, minimization of enstrophy, etc governed by the two dimensional CBF equations with the absorption exponent <inline-formula><tex-math id="M1">\begin{document}$ r = 1,2 $\end{document}</tex-math></inline-formula> and <inline-formula><tex-math id="M2">\begin{document}$ 3 $\end{document}</tex-math></inline-formula>. We show the existence of an optimal solution and the first order necessary conditions of optimality for such optimal control problems in terms of the Euler-Lagrange system. Furthermore, for the case <inline-formula><tex-math id="M3">\begin{document}$ r = 3 $\end{document}</tex-math></inline-formula>, we show the second order necessary and sufficient conditions of optimality. We also investigate an another control problem which is similar to that of the data assimilation problems in meteorology of obtaining unknown initial data, when the system under consideration is 2D CBF equations, using optimal control techniques.</p>

Strong Solvability of a Variational Data Assimilation Problem for the Primitive Equations of Large-Scale Atmosphere and Ocean Dynamics

For the primitive equations of large-scale atmosphere and ocean dynamics, we study the problem of determining by means of a variational data assimilation algorithm initial conditions that generate strong solutions which minimize the distance to a given set of time-distributed observations. We suggest a modification of the adjoint algorithm whose novel elements is to use norms in the variational cost functional that reflects the H^1-regularity of strong solutions of the primitive equations. For such a cost functional, we prove the existence of minima and a first-order adjoint condition for strong solutions that provides the basis for computing these minima. We prove the local convergence of a gradient-based descent algorithm to optimal initial conditions using the second-order adjoint primitive equations. The algorithmic modifications due to the H^1-norms are straightforwardly to implement into a variational algorithm that employs the standard L^2-metrics.

P-Kernel Stein Variational Gradient Descent for Data Assimilation and History Matching

A Bayesian method of inference known as “Stein variational gradient descent” was recently implemented for data assimilation problems, under the heading of “mapping particle filter”. In this manuscript, the algorithm is applied to another type of geoscientific inversion problems, namely history matching of petroleum reservoirs. In order to combat the curse of dimensionality, the commonly used Gaussian kernel, which defines the solution space, is replaced by a p-kernel. In addition, the ensemble gradient approximation used in the mapping particle filter is rectified, and the data assimilation experiments are re-run with more relevant settings and comparisons. Our experimental results in data assimilation are rather disappointing. However, the results from the subsurface inverse problem show more promise, especially as regards the use of p-kernels.

Reduced Order Models for Optimal Flow Control

AbstractData assimilation models allow to fill the gap between numerical simulations and experimental data. Optimal control problems governed by parametrized partial differential equations PDE(µ) is suited for this kind of application, where you want to track problem solutions towards known quantities, given by data collections or previous knowledge. Still, the computational effort increases when one has to deal with nonlinear time‐dependent governing equations. Reduced order methods are an effective approach to solve data assimilation problems in a reliable and faster way. We apply the POD‐Galerkin methodology in environmental marine sciences where different parameters describe several physical configurations. We present a nonlinear time‐dependent tracking problem for velocity‐height solutions of shallow water equations.

Restoring the Boundary Condition and the Initial State in the Variational Data Assimilation Problem for the Black Sea Dynamics Model

In the present paper, we formulate an inverse problem of assimilation of sea surface temperature for a sea thermodynamics model aimed at the reconstruction of the heat flux and the initial state. We assume, that the unique function which is obtained by observation data processing is the function of sea surface temperature (SST). Numerical experiments on restoring the surface heat flux and the initial state for the system of the Black Sea primitive equation hydrodynamics model (temperature, salinity, velocity, and sea surface level) with assimilation procedure were carried. The results of the numerical experiments are presented.

Geophysics-Based Fluid-Facies Predictions Using Ensemble Updating of Binary State Vectors

Fluid flow simulations are commonly used to predict the fluid displacement in subsurface reservoirs; however, model validation is challenging due to the lack of direct measurements. Geophysical data can be used to monitor the displacement of the fluid front. The updating of the fluid front location in two-phase flow problems based on time-lapse geophysical data can be formulated as an inverse problem, specifically a data assimilation problem, where the state is a vector of binary variables representing the fluid-facies and the observations are measurements of continuous geophysical properties, such as electrical or elastic properties. In geosciences, many data assimilation problems are solved using ensemble-based methods relying on the Kalman filter approach. However, for discrete variables, such approaches cannot be applied due to the Gaussian-linear assumption. An innovative approach for mixed discrete-continuous problems based on ensemble updating of binary state vectors is presented for fluid-facies prediction problems with time-lapse geophysical properties. The proposed inversion method is demonstrated in a synthetic two-dimensional simulation example where water is injected into a reservoir and hydrocarbon is produced. Resistivity values obtained from controlled-source electromagnetic data are assumed to be available at different times. According to the results, the proposed inversion method is to a large extent able to reproduce the true underlying binary field of fluid-facies.

Behavior of the iterative ensemble-based variational method in nonlinear problems

Abstract. The behavior of the iterative ensemble-based data assimilation algorithm is discussed. The ensemble-based method for variational data assimilation problems, referred to as the 4D ensemble variational method (4DEnVar), is a useful tool for data assimilation problems. Although the 4DEnVar is derived based on a linear approximation, highly uncertain problems, in which system nonlinearity is significant, are solved by applying this method iteratively. However, the ensemble-based methods basically seek the solution within a lower-dimensional subspace spanned by the ensemble members. It is not necessarily trivial how high-dimensional problems can be solved with the ensemble-based algorithm which employs the lower-dimensional approximation based on the ensemble. In the present study, an ensemble-based iterative algorithm is reformulated to allow us to analyze its behavior in high-dimensional nonlinear problems. The conditions for monotonic convergence to a local maximum of the objective function are discussed in a high-dimensional context. It is shown that the ensemble-based algorithm can solve high-dimensional problems by distributing the ensemble in different subspace at each iteration. The findings as the results of the present study were also experimentally supported.

Parallel framework for dynamic domain decomposition of data assimilation problems: a case study on Kalman Filter algorithm

We focus on Partial Differential Equation (PDE) based Data Assimilatio problems (DA) solved by means of variational approaches and Kalman filter algorithm. Recently, we presented a Domain Decomposition framework (we call it DD-DA, for short) performing a decomposition of the whole physical domain along space and time directions, and joining the idea of Schwarz' methods and parallel in time approaches. For effective parallelization of DD-DA algorithms, the computational load assigned to subdomains must be equally distributed. Usually computational cost is proportional to the amount of data entities assigned to partitions. Good quality partitioning also requires the volume of communication during calculation to be kept at its minimum. In order to deal with DD-DA problems where the observations are nonuniformly distributed and general sparse, in the present work we employ a parallel load balancing algorithm based on adaptive and dynamic defining of boundaries of DD -- which is aimed to balance workload according to data location. We call it DyDD. As the numerical model underlying DA problems arising from the so-called discretize-then-optimize approach is the constrained least square model (CLS), we will use CLS as a reference state estimation problem and we validate DyDD on different scenarios.

Numerical solution of the problem of variational data assimilation to restore heat fluxes and initial state for the ocean thermodynamics model

Abstract For the model of ocean thermodynamics developed at the Institute of Numerical Mathematics of RAS, the problem of variational data assimilation is considered in order to restore heat fluxes on the ocean surface and initial state of the model. Iterative solution algorithms are proposed for the optimality system, justification of these algorithms is given based on properties of the control operator. The results of numerical experiments for the model of the Black Sea dynamics are presented.

Parallel Dynamic Domain Decomposition in Space - Time for Data Assimilation problems

In the present work we employ a load balancing scheme involving an adaptive and dynamic workload redistribution both along Space and Time directions for solving Data Assimilation problems where the observations are non-uniformly distributed, general sparse and its distribution changes during the time. We will consider the Constrained Least Square model (CLS) as prototype of Data Assimilation problems and we will validate the proposed approach on different configurations. Validation is performed using Parallel Computing Toolbox of MATLABR2013a on high performance hybrid computing architecture.

Deep-learning-based surrogate flow modeling and geological parameterization for data assimilation in 3D subsurface flow

Data assimilation in subsurface flow systems is challenging due to the large number of flow simulations often required, and by the need to preserve geological realism in the calibrated (posterior) models. In this work we present a deep-learning-based surrogate model for two-phase flow in 3D subsurface formations. This surrogate model, a 3D recurrent residual U-Net (referred to as recurrent R-U-Net), consists of 3D convolutional and recurrent (convLSTM) neural networks, designed to capture the spatial–temporal information associated with dynamic subsurface flow systems. A CNN-PCA procedure (convolutional neural network post-processing of principal component analysis) for parameterizing complex 3D geomodels is also described. This approach represents a simplified version of a recently developed supervised-learning-based CNN-PCA framework. The recurrent R-U-Net is trained on the simulated dynamic 3D saturation and pressure fields for a set of random ‘channelized’ geomodels (generated using 3D CNN-PCA). Detailed flow predictions demonstrate that the recurrent R-U-Net surrogate model provides accurate results for dynamic states and well responses for new geological realizations, along with accurate flow statistics for an ensemble of new geomodels. The 3D recurrent R-U-Net and CNN-PCA procedures are then used in combination for a challenging data assimilation problem involving a channelized system. Two different algorithms, namely rejection sampling and an ensemble-based method, are successfully applied. The overall methodology described in this paper may enable the assessment and refinement of data assimilation procedures for a range of realistic and challenging subsurface flow problems.

A Nonmonotone Matrix-Free Algorithm for Nonlinear Equality-Constrained Least-Squares Problems

Least squares form one of the most prominent classes of optimization problems, with numerous applications in scientific computing and data fitting. When such formulations aim at modeling complex systems, the optimization process must account for nonlinear dynamics by incorporating constraints. In addition, these systems often incorporate a large number of variables, which increases the difficulty of the problem, and motivates the need for efficient algorithms amenable to large-scale implementations. In this paper, we propose and analyze a Levenberg-Marquardt algorithm for nonlinear least squares subject to nonlinear equality constraints. Our algorithm is based on inexact solves of linear least-squares problems, that only require Jacobian-vector products. Global convergence is guaranteed by the combination of a composite step approach and a nonmonotone step acceptance rule. We illustrate the performance of our method on several test cases from data assimilation and inverse problems: our algorithm is able to reach the vicinity of a solution from an arbitrary starting point, and can outperform the most natural alternatives for these classes of problems.

Using Deep Learning to Improve Ensemble Smoother: Applications to Subsurface Characterization

AbstractEnsemble smoother (ES) has been widely used in various research fields to reduce the uncertainty of the system‐of‐interest. However, the commonly adopted ES method that employs the Kalman formula, that is, ES(K), does not perform well when the probability distributions involved are non‐Gaussian. To address this issue, we suggest to use deep learning (DL) to derive an alternative analysis scheme for ES in non‐Gaussian data assimilation problems. Here we show that the DL‐based ES method, that is, ES(DL), is more general and flexible. In this new scheme, a high volume of training data is generated from a relatively small‐sized ensemble of model parameters and simulation outputs, and possible non‐Gaussian features can be preserved in the training data and captured by an adequate DL model. This new variant of ES is tested in two subsurface characterization problems with or without the Gaussian assumption. Results indicate that ES(DL) can produce similar (in the Gaussian case) or even better (in the non‐Gaussian case) results compared to those from ES(K). The success of ES(DL) comes from the power of DL in extracting complex (including non‐Gaussian) features and learning nonlinear relationships from massive amounts of training data. Although in this work we only apply the ES(DL) method in parameter estimation problems, the proposed idea can be conveniently extended to analysis of model structural uncertainty and state estimation in real‐time forecasting problems.

Error covariance tuning in variational data assimilation: application to an operating hydrological model

Because the true state of complex physical systems is out of reach for real-world data assimilation problems, error covariances are uncertain and their specification remains very challenging. These error covariances are crucial ingredients for the proper use of data assimilation methods and for an effective quantification of the a posteriori errors of the state estimation. Therefore, the estimation of these covariances often involves at first a chosen specification of the matrices, followed by an adaptive tuning to correct their initial structure. In this paper, we propose a flexible combination of existing covariance tuning algorithms, including both online and offline procedures. These algorithms are applied in a specific order such that the required assumption of current tuning algorithms are fulfilled, at least partially, by the application of the ones at the previous steps. We use our procedure to tackle the problem of a multivariate and spatially-distributed hydrological model based on a precipitation-flow simulator with real industrial data. The efficiency of different algorithmic schemes is compared using real data with both quantitative and qualitative analysis. Numerical results show that these proposed algorithmic schemes improve significantly short-range flow forecast. Among the several tuning methods tested, recently developed CUTE and PUB algorithms are in the lead both in terms of history matching and forecast.