Data Assimilation Problems Research Articles

Machine Learning: Deepest Learning as Statistical Data Assimilation Problems.

We formulate an equivalence between machine learning and the formulation of statistical data assimilation as used widely in physical and biological sciences. The correspondence is that layer number in a feedforward artificial network setting is the analog of time in the data assimilation setting. This connection has been noted in the machine learning literature. We add a perspective that expands on how methods from statistical physics and aspects of Lagrangian and Hamiltonian dynamics play a role in how networks can be trained and designed. Within the discussion of this equivalence, we show that adding more layers (making the network deeper) is analogous to adding temporal resolution in a data assimilation framework. Extending this equivalence to recurrent networks is also discussed. We explore how one can find a candidate for the global minimum of the cost functions in the machine learning context using a method from data assimilation. Calculations on simple models from both sides of the equivalence are reported. Also discussed is a framework in which the time or layer label is taken to be continuous, providing a differential equation, the Euler-Lagrange equation and its boundary conditions, as a necessary condition for a minimum of the cost function. This shows that the problem being solved is a two-point boundary value problem familiar in the discussion of variational methods. The use of continuous layers is denoted "deepest learning." These problems respect a symplectic symmetry in continuous layer phase space. Both Lagrangian versions and Hamiltonian versions of these problems are presented. Their well-studied implementation in a discrete time/layer, while respecting the symplectic structure, is addressed. The Hamiltonian version provides a direct rationale for backpropagation as a solution method for a certain two-point boundary value problem.

Reduced basis approximation and a posteriori error bounds for 4D-Var data assimilation

We propose a certified reduced basis approach for the strong- and weak-constraint four-dimensional variational (4D-Var) data assimilation problem for a parametrized PDE model. While the standard strong-constraint 4D-Var approach uses the given observational data to estimate only the unknown initial condition of the model, the weak-constraint 4D-Var formulation additionally provides an estimate for the model error and thus can deal with imperfect models. Since the model error is a distributed function in both space and time, the 4D-Var formulation leads to a large-scale optimization problem for every given parameter instance of the PDE model. To solve the problem efficiently, various reduced order approaches have therefore been proposed in the recent past. Here, we employ the reduced basis method to generate reduced order approximations for the state, adjoint, initial condition, and model error. Our main contribution is the development of efficiently computable a posteriori upper bounds for the error of the reduced basis approximation with respect to the underlying high-dimensional 4D-Var problem. Numerical results are conducted to test the validity of our approach.

Adjoint equations in variational data assimilation problems

Abstract The problem of variational data assimilation for an evolution model is considered with the aim to identify the initial condition. The solvability of the optimality system is studied. Based on the adjoint equations, iterative algorithms for solving the problem are developed and justified.

A note on preconditioning weighted linear least‐squares, with consequences for weakly constrained variational data assimilation

The effect of preconditioning linear weighted least‐squares using an approximation of the model matrix is analyzed. The aim is to investigate from a theoretical point of view the inefficiencies of this approach as observed in the application of the weakly constrained 4D‐Var algorithm in geosciences. Bounds on the eigenvalues of the preconditioned system matrix are provided. It highlights the interplay of the eigenstructures of both the model and weighting matrices: maintaining a low bound on the eigenvalues of the preconditioned system matrix requires an approximation error of the model matrix which compensates for the condition number of the weighting matrix. A low‐dimension analytical example is given illustrating the resulting potential inefficiency of such preconditioners. The consequences of these results in the context of the state formulation of the weakly constrained 4D‐Var data assimilation problem are discussed. It is shown that the common approximations of the tangent linear model which maintain parallelization‐in‐time properties (identity or null matrix) can result in large bounds on the eigenvalues of the preconditioned matrix system.

Stability of the Optimal Solution to a Problem of Variational Assimilation with Error Covariance Matrices of Observational Data for a Sea Thermodynamics Model

A mathematical model of sea thermodynamics developed at the Institute of Numerical Mathematics of the Russian Academy of Sciences is considered. A problem of variational assimilation of daily-averaged sea surface temperature (SST) data with observational error covariance matrices is formulated and investigated. On the basis of variational assimilation of satellite observation data, an inverse problem of restoring the heat flux on the sea surface is solved. The stability of the optimal solution to the problem of variational data assimilation is studied, and the results of numerical experiments with the model for Baltic Sea dynamics are presented.

The conditioning of least‐squares problems in variational data assimilation

SummaryIn variational data assimilation a least‐squares objective function is minimised to obtain the most likely state of a dynamical system. This objective function combines observation and prior (or background) data weighted by their respective error statistics. In numerical weather prediction, data assimilation is used to estimate the current atmospheric state, which then serves as an initial condition for a forecast. New developments in the treatment of observation uncertainties have recently been shown to cause convergence problems for this least‐squares minimisation. This is important for operational numerical weather prediction centres due to the time constraints of producing regular forecasts. The condition number of the Hessian of the objective function can be used as a proxy to investigate the speed of convergence of the least‐squares minimisation. In this paper we develop novel theoretical bounds on the condition number of the Hessian. These new bounds depend on the minimum eigenvalue of the observation error covariance matrix and the ratio of background error variance to observation error variance. Numerical tests in a linear setting show that the location of observation measurements has an important effect on the condition number of the Hessian. We identify that the conditioning of the problem is related to the complex interactions between observation error covariance and background error covariance matrices. Increased understanding of the role of each constituent matrix in the conditioning of the Hessian will prove useful for informing the choice of correlated observation error covariance matrix and observation location, particularly for practical applications.

Hessian-based covariance approximations in variational data assimilation

AbstractThe problem of variational data assimilation (estimation) for a nonlinear model is considered in general operator formulation. Hessian-based methods are presented to compute the estimation error covariances. The importance of dynamic formulation and the role of the Hessian and its inverse are discussed.

Ensemble Kalman filter for the reconstruction of the Earth's mantle circulation

Abstract. Recent advances in mantle convection modeling led to the release of a new generation of convection codes, able to self-consistently generate plate-like tectonics at their surface. Those models physically link mantle dynamics to surface tectonics. Combined with plate tectonic reconstructions, they have the potential to produce a new generation of mantle circulation models that use data assimilation methods and where uncertainties in plate tectonic reconstructions are taken into account. We provided a proof of this concept by applying a suboptimal Kalman filter to the reconstruction of mantle circulation (Bocher et al., 2016). Here, we propose to go one step further and apply the ensemble Kalman filter (EnKF) to this problem. The EnKF is a sequential Monte Carlo method particularly adapted to solve high-dimensional data assimilation problems with nonlinear dynamics. We tested the EnKF using synthetic observations consisting of surface velocity and heat flow measurements on a 2-D-spherical annulus model and compared it with the method developed previously. The EnKF performs on average better and is more stable than the former method. Less than 300 ensemble members are sufficient to reconstruct an evolution. We use covariance adaptive inflation and localization to correct for sampling errors. We show that the EnKF results are robust over a wide range of covariance localization parameters. The reconstruction is associated with an estimation of the error, and provides valuable information on where the reconstruction is to be trusted or not.

Implicit surrogate models for trust region based methods

In this paper, theoretical and practical aspects of Taylor based surrogate models for trust region (TR) optimization are discussed. We propose two surrogate models: one of them based on convex combinations of first order Taylor polynomials while the other one is based on that of quadratic Taylor expansions. During optimization steps of TR methods, the proposed models are built as follows: samples from the cost function are taken about the current solution and then, a linear or a quadratic Taylor polynomial is built about each sampled point. All Taylor expansions are convexly combined by using radial basis functions in order to propose a surrogate model for the optimization step. Theoretically, we prove that, the proposed models are fully linear or fully quadratic depending on whether they are built based on convex combinations of linear or quadratic Taylor expansions, respectively. This guaranties that, regardless the initial guess of the iterative process, the optimization method will converge to local solutions where at least first order optimality conditions are satisfied. Discussions on how to solve the surrogate optimization problems among iterations are presented as well. Experimental results are carried out making use of a sequential data assimilation problem. The results reveal that, the use of our proposed models in such context can improve the quality of analysis states in terms of root-mean-square-error values while uncertainties can be decreased across different assimilation cycles.

A low-rank approach to the solution of weak constraint variational data assimilation problems

Weak constraint four-dimensional variational data assimilation is an important method for incorporating data (typically observations) into a model. The linearised system arising within the minimisation process can be formulated as a saddle point problem. A disadvantage of this formulation is the large storage requirements involved in the linear system. In this paper, we present a low-rank approach which exploits the structure of the saddle point system using techniques and theory from solving large scale matrix equations. Numerical experiments with the linear advection–diffusion equation, and the non-linear Lorenz-95 model demonstrate the effectiveness of a low-rank Krylov subspace solver when compared to a traditional solver.

Data assimilation in Parker’s dynamo model

Application of the extended Kalman filter in the data assimilation problem of the one-dimensional (1D) Parker dynamo model is considered for two extreme cases: (1) when the magnetic field observations are taken uniformly across the entire surface of the liquid core of the Earth and (2) when the observations are taken at one spatial point. The algorithm allows the model solution with arbitrary initial conditions to be modified based on observations. It is shown that the redundancy of the noised data can reduce the efficiency of the recovery of the solution. Considerations on the applicability of data assimilation for the case of a higher dimension and a solution’s convergence dependent on the density of the flow of information assimilated are suggested.

Primal-Dual Mixed Finite Element Methods for the Elliptic Cauchy Problem

consider primal-dual mixed finite element methods for the solution of the elliptic Cauchy problem, or other related data assimilation problems. The method has a local conservation property. We deri ...

Asymptotic Forecast Uncertainty and the Unstable Subspace in the Presence of Additive Model Error

It is well understood that dynamic instability is among the primary drivers of forecast uncertainty in chaotic, physical systems. Data assimilation techniques have been designed to exploit this phenomenon, reducing the effective dimension of the data assimilation problem to the directions of rapidly growing errors. Recent mathematical work has, moreover, provided formal proofs of the central hypothesis of the assimilation in the unstable subspace methodology of Anna Trevisan and her collaborators: for filters and smoothers in perfect, linear, Gaussian models, the distribution of forecast errors asymptotically conforms to the unstable-neutral subspace. Specifically, the column span of the forecast and posterior error covariances asymptotically align with the span of backward Lyapunov vectors with nonnegative exponents. Earlier mathematical studies have focused on perfect models, and this current work now explores the relationship between dynamical instability, the precision of observations, and the evolution of forecast error in linear models with additive model error. We prove bounds for the asymptotic uncertainty, explicitly relating the rate of dynamical expansion, model precision, and observational accuracy. Formalizing this relationship, we provide a novel, necessary criterion for the boundedness of forecast errors. Furthermore, we numerically explore the relationship between observational design, dynamical instability, and filter boundedness. Additionally, we include a detailed introduction to the multiplicative ergodic theorem and to the theory and construction of Lyapunov vectors. While forecast error in the stable subspace may not generically vanish, we show that even without filtering, uncertainty remains uniformly bounded due its dynamical dissipation. However, the continuous reinjection of uncertainty from model errors may be excited by transient instabilities in the stable modes of high variance, rendering forecast uncertainty impractically large. In the context of ensemble data assimilation, this requires rectifying the rank of the ensemble-based gain to account for the growth of uncertainty beyond the unstable and neutral subspace, additionally correcting stable modes with frequent occurrences of positive local Lyapunov exponents that excite model errors.

DATA ASSIMILATION FOR NAVIER-STOKES USING THE LEAST-SQUARES FINITE-ELEMENT METHOD

We investigate theoretically and numerically the use of the Least-Squares Finite-element method (LSFEM) to approach data-assimilation problems for the steady-state, incompressible Navier-Stokes equations. Our LSFEM discretization is based on a stress-velocity-pressure (S-V-P) first-order formulation, using discrete counterparts of the Sobolev spaces $H({\rm div}) \times H^1 \times L^2$ respectively. Resolution of the system is via minimization of a least-squares functional representing the magnitude of the residual of the equations. A simple and immediate approach to extend this solver to data-assimilation is to add a data-discrepancy term to the functional. Whereas most data-assimilation techniques require a large number of evaluations of the forward-simulations and are therefore very expensive, the approach proposed in this work uniquely has the same cost as a single forward run. However, the question arises: what is the statistical model implied by this choice? We answer this within the Bayesian framework, establishing the latent background covariance model and the likelihood. Further we demonstrate that - in the linear case - the method is equivalent to application of the Kalman filter, and derive the posterior covariance. We practically demonstrate the capabilities of our method on a backward-facing step case. Our LSFEM formulation (without data) is shown to have good approximation quality, even on relatively coarse meshes - in particular with respect to mass-conservation and reattachment location. Adding limited velocity measurements from experiment, we show that the method is able to correct for discretization error on very coarse meshes, as well as correct for the influence of unknown and uncertain boundary-conditions.

Nonlinear Least Squares En4DVar to 4DEnVar Methods for Data Assimilation: Formulation, Analysis, and Preliminary Evaluation

Abstract The En4DVar method is designed to combine the flow-dependent statistical covariance information of EnKF into the traditional 4DVar method. However, the En4DVar method is still hampered by its strong dependence on the adjoint model of the underlying forecast model and by its complexity, maintenance requirements, and the high cost of computer implementation and simulation. The primary goal of this paper is to propose an alternative approach to overcome the main difficulty of the En4DVar method caused by the use of adjoint models. The proposed approach, the nonlinear least squares En4DVar (NLS-En4DVar) method, begins with rewriting the standard En4DVar formulation into a nonlinear least squares problem, which is followed by solving the resulting NLS problem by a Gauss–Newton iterative method. To reduce the computational and implementation complexity of the proposed NLS-En4DVar method, a few variants of the new method are proposed; these modifications make the model cheaper and easier to use than the full NLS-En4DVar method at the expense of reduced accuracy. Furthermore, an improved iterative method based on the comprehensive analysis on the above NLSi-En4DVar family of methods is also proposed. These proposed NLSi-En4DVar methods provide more flexible choices of the computational capabilities for the broader and more realistic data assimilation problems arising from various applications. The pros and cons of the proposed NLSi-En4DVar family of methods are further examined in the paper and their relationships and performance are also evaluated by several sets of numerical experiments based on the Lorenz-96 model and the Advanced Research WRF (ARW) Model, respectively.

An Adaptive Parametrized-Background Data-Weak approach to variational data assimilation

We present an Adaptive Parametrized-Background Data-Weak (APBDW) approach to the steady-state variational data assimilation (state estimation) problem for systems modeled by partial differential equations. The variational formulation is based on the Tikhonov regularization of the PBDW formulation [Y. Maday, A.T. Patera, J.D. Penn and M. Yano, Int. J. Numer. Meth. Eng. 102 (2015) 933–965] for pointwise noisy measurements. We propose an adaptive procedure based on a posteriori estimates of the L 2 state-estimation error to improve performance. We also present a priori estimates for the L 2 state-estimation error that motivate the approach and guide the adaptive procedure. We provide numerical experiments for a synthetic acoustic problem to illustrate the different elements of the methodology, and we consider an experimental thermal patch configuration to demonstrate the applicability of our approach to real physical systems.

Green\u2019s function for the boundary value problem of the static Klein-Gordon equation stated on a rectangular region and its convergence analysis

This study is motivated by solving the inverse boundary problem of the static Klein-Gordon equation (SKGE), which usually occurs in data assimilation problems. For the purpose of obtaining boundary conditions, the pro forma solution of the problem is provided by using Green’s function. The representation in double series of Green’s function for the SKGE on a rectangular region is obtained by means of the method of images. Convergence analysis shows that the representation is uniformly convergent, which is computer friendly, and can be applied to approximate computations.

A velocity tracking approach for the data assimilation problem in blood flow simulations

Several advances have been made in data assimilation techniques applied to blood flow modeling. Typically, idealized boundary conditions, only verified in straight parts of the vessel, are assumed. We present a general approach, on the basis of a Dirichlet boundary control problem, that may potentially be used in different parts of the arterial system. The relevance of this method appears when computational reconstructions of the 3D domains, prone to be considered sufficiently extended, are either not possible, or desirable, because of computational costs. On the basis of taking a fully unknown velocity profile as the control, the approach uses a discretize then optimize methodology to solve the control problem numerically. The methodology is applied to a realistic 3D geometry representing a brain aneurysm. The results show that this data assimilation approach may be preferable to a pressure control strategy and that it can significantly improve the accuracy associated to typical solutions obtained using idealized velocity profiles.

Sensitivity with respect to observations in variational data assimilation

Abstract The problem of variational data assimilation for a nonlinear evolution model is formulated as an optimal control problem to find the initial condition function (analysis). The observation data, and hence the optimal solution, may contain uncertainties. A response function is considered as a functional of the optimal solution after assimilation. Based on the second-order adjoint techniques, the sensitivity of the response function to the observation data is studied. The gradient of the response function is related to the solution of a non-standard problem involving the coupled system of direct and adjoint equations. The solvability of the non-standard problem is studied, based on the Hessian of the original cost function. An algorithm to compute the gradient of the response function with respect to observations is developed and justified.

On the variational data assimilation problem solving and sensitivity analysis

We consider the Variational Data Assimilation (VarDA) problem in an operational framework, namely, as it results when it is employed for the analysis of temperature and salinity variations of data collected in closed and semi closed seas. We present a computing approach to solve the main computational kernel at the heart of the VarDA problem, which outperforms the technique nowadays employed by the oceanographic operative software. The new approach is obtained by means of Tikhonov regularization. We provide the sensitivity analysis of this approach and we also study its performance in terms of the accuracy gain on the computed solution. We provide validations on two realistic oceanographic data sets.