Assimilation Problem Research Articles

Sensitivity of Functionals of Solution to a Variational Data Assimilation Problem with Simultaneous Reconstruction of Heat Fluxes and Initial State for a Sea Thermodynamics Model

A problem of variational assimilation of data is considered in order to simultaneously reconstruct sea surface heat fluxes and the initial state of a mathematical model of sea thermodynamics developed at the Institute of Numerical Mathematics of the Russian Academy of Sciences. The sensitivity of some solution functionals to observation data in the variational assimilation problem is investigated, and results of numerical experiments with the model applied to Baltic Sea dynamics are presented.

Numerical linear algebra in data assimilation

AbstractData assimilation is a method that combines observations (ie, real world data) of a state of a system with model output for that system in order to improve the estimate of the state of the system and thereby the model output. The model is usually represented by a discretized partial differential equation. The data assimilation problem can be formulated as a large scale Bayesian inverse problem. Based on this interpretation we will derive the most important variational and sequential data assimilation approaches, in particular three‐dimensional and four‐dimensional variational data assimilation (3D‐Var and 4D‐Var) and the Kalman filter. We will then consider more advanced methods which are extensions of the Kalman filter and variational data assimilation and pay particular attention to their advantages and disadvantages. The data assimilation problem usually results in a very large optimization problem and/or a very large linear system to solve (due to inclusion of time and space dimensions). Therefore, the second part of this article aims to review advances and challenges, in particular from the numerical linear algebra perspective, within the various data assimilation approaches.

Methods of variational data assimilation with application to problems of hydrothermodynamics of marine water areas

AbstractA series of problems related to the class of inverse problems of ocean hydrothermodynamics and problems of variational data assimilation are formulated in the present paper. We propose methods for solving the problems studied here and present results of numerical experiments.

Background Error in WRF Model

WRF model have been tuned and tested over Georgia’s territory for years. First time in Georgia theprocess of data assimilation in Numerical weather prediction is developing. This work presents how forecasterror statistics appear in the data assimilation problem through the background error covariance matrix – B, wherethe variances and correlations associated with model forecasts are estimated. Results of modeling of backgrounderror covariance matrix for control variables using WRF model over Georgia with desired domain configurationare discussed and presented. The modeling was implemented in two different 3DVAR systems (WRFDA andGSI) and results were checked by pseudo observation benchmark cases using also default global and regional BEmatrixes. The mathematical and physical properties of the covariances are also reviewed.

An offline framework for high-dimensional ensemble Kalman filters to reduce the time to solution

Abstract. The high computational resources and the time-consuming IO (input/output) are major issues in offline ensemble-based high-dimensional data assimilation systems. Bearing these in mind, this study proposes a sophisticated dynamically running job scheme as well as an innovative parallel IO algorithm to reduce the time to solution of an offline framework for high-dimensional ensemble Kalman filters. The dynamically running job scheme runs as many tasks as possible within a single job to reduce the queuing time and minimize the overhead of starting and/or ending a job. The parallel IO algorithm reads or writes non-overlapping segments of multiple files with an identical structure to reduce the IO times by minimizing the IO competitions and maximizing the overlapping of the MPI (Message Passing Interface) communications with the IO operations. Results based on sensitive experiments show that the proposed parallel IO algorithm can significantly reduce the IO times and have a very good scalability, too. Based on these two advanced techniques, the offline and online modes of ensemble Kalman filters are built based on PDAF (Parallel Data Assimilation Framework) to comprehensively assess their efficiencies. It can be seen from the comparisons between the offline and online modes that the IO time only accounts for a small fraction of the total time with the proposed parallel IO algorithm. The queuing time might be less than the running time in a low-loaded supercomputer such as in an operational context, but the offline mode can be nearly as fast as, if not faster than, the online mode in terms of time to solution. However, the queuing time is dominant and several times larger than the running time in a high-loaded supercomputer. Thus, the offline mode is substantially faster than the online mode in terms of time to solution, especially for large-scale assimilation problems. From this point of view, results suggest that an offline ensemble Kalman filter with an efficient implementation and a high-performance parallel file system should be preferred over its online counterpart for intermittent data assimilation in many situations.

Numerical Modeling of Marine Circulation with 4D Variational Data Assimilation

The technology is presented for modeling and prediction of marine hydrophysical fields based on the 4D variational data assimilation technique developed at the Marchuk Institute of Numerical Mathematics, Russian Academy of Sciences (INM RAS). The technology is based on solving equations of marine hydrodynamics using multicomponent splitting, thereby solving an optimality system that includes adjoint equations and covariance matrices of observation errors. The hydrodynamic model is described by primitive equations in the sigma-coordinate system, which is solved by finite-difference methods. The technology includes original algorithms for solving the problems of variational data assimilation using modern iterative processes with a special choice of iterative parameters. The methods and technology are illustrated by the example of solving the problem of circulation of the Baltic Sea with 4D variational data assimilation of sea surface temperature information.

Maximum Principle for Some Optimal Control Problems Governed by 2D Nonlocal Cahn–Hillard–Navier–Stokes Equations

This work concerns some optimal control problems associated with the evolution of two isothermal, incompressible, immiscible fluids in a two-dimensional bounded domain. The Cahn–Hilliard–Navier–Stokes model consists of a Navier–Stokes equation governing the fluid velocity field coupled with a convective Cahn–Hilliard equation for the relative concentration of one of the fluids. A distributed optimal control problem is formulated as the minimization of a cost functional subject to the controlled nonlocal Cahn–Hilliard–Navier–Stokes equations. We establish the first-order necessary conditions of optimality by proving the Pontryagin maximum principle for optimal control of such system via the seminal Ekeland variational principle. The optimal control is characterized using the adjoint variable. We also study an another optimal control problem of finding the unknown optimal initial data. Optimal data initialization problem is also known as the data assimilation problems in meteorology, where determining the correct initial condition is very crucial for the future predictions.

Variational Data Assimilation in Problems of Modeling Hydrophysical Fields in Open Water Areas

The formulation of boundary conditions at liquid (open) boundaries is a topical problem in mathematically modeling the hydrothermodynamics of open water areas. Variational data assimilation is one method allowing one to take into account liquid boundaries in models. According to the approach considered in this paper, observational data at a certain time are given and the problem is treated as an inverse one with open boundary flows as additional unknowns. This paper presents a formulation of the general problem of the variational assimilation of observational data for a model of the hydrothermodynamics of open water areas based on the splitting method. Algorithms for the variational assimilation of temperature and sea-level data at the liquid boundary are formulated and the results of numerical experiments on the use of the algorithms in the Baltic Sea circulation model are presented.

A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems

A deep-learning-based surrogate model is developed and applied for predicting dynamic subsurface flow in channelized geological models. The surrogate model is based on deep convolutional and recurrent neural network architectures, specifically a residual U-Net and a convolutional long short term memory recurrent network. Training samples entail global pressure and saturation maps, at a series of time steps, generated by simulating oil-water flow in many (1500 in our case) realizations of a 2D channelized system. After training, the ‘recurrent R-U-Net’ surrogate model is shown to be capable of predicting accurate dynamic pressure and saturation maps and well rates (e.g., time-varying oil and water rates at production wells) for new geological realizations. Assessments demonstrating high surrogate-model accuracy are presented for an individual geological realization and for an ensemble of 500 test geomodels. The surrogate model is then used for the challenging problem of data assimilation (history matching) in a channelized system. For this study, posterior reservoir models are generated using the randomized maximum likelihood method, with the permeability field represented using the recently developed CNN-PCA parameterization. The flow responses required during the data assimilation procedure are provided by the recurrent R-U-Net. The overall approach is shown to lead to substantial reduction in prediction uncertainty. High-fidelity numerical simulation results for the posterior geomodels (generated by the surrogate-based data assimilation procedure) are shown to be in essential agreement with the recurrent R-U-Net predictions. Comparisons to simulation-based data assimilation results further highlight the accuracy and applicability of the recurrent R-U-Net, and suggest that it may enable the use of more formal posterior sampling methods in realistic problems.

A finite element data assimilation method for the wave equation

We design a primal-dual stabilized finite element method for the numerical approximation of a data assimilation problem subject to the acoustic wave equation. For the forward problem, piecewise affine, continuous, finite element functions are used for the approximation in space and backward differentiation is used in time. Stabilizing terms are added on the discrete level. The design of these terms is driven by numerical stability and the stability of the continuous problem, with the objective of minimizing the computational error. Error estimates are then derived that are optimal with respect to the approximation properties of the numerical scheme and the stability properties of the continuous problem. The effects of discretizing the (smooth) domain boundary and other perturbations in data are included in the analysis.

A bilevel learning approach for optimal observation placement in variational data assimilation

In this paper we propose a bilevel optimization approach for the placement of space and time observations in variational data assimilation problems. Within the framework of supervised learning, we consider a bilevel problem where the lower-level task is the variational reconstruction of the initial condition of a semilinear system, and the upper-level problem solves the optimal placement with help of a sparsity inducing function. Due to the pointwise nature of the observations, an optimality system with regular Borel measures is obtained as necessary optimality condition for the lower-level problem. The latter is then considered as constraint for the upper-level instance, yielding an optimization problem constrained by a multi-state system with measures. We demonstrate existence of Lagrange multipliers and derive a necessary optimality system characterizing the optimal solutions of the bilevel problem. The numerical solution is carried out also on two levels. The lower-level problem is solved using a standard BFGS method, while the upper-level one is solved by means of a projected BFGS algorithm, based on the estimation of -active sets. Finally some numerical experiments are presented to illustrate the main features of our approach.

Stably numerical solving inverse boundary value problem for data assimilation

In remote sensing data assimilation, inverse methods are often used to retrieve the boundary conditions from some of the observations in the interior scanning region. Needless to say, the inverse boundary value problem (IBVP) is ill-conditioned in general. Ultimately, the data assimilation must include mechanisms that enable them to overcome the numerical instability for solving IBVP. In this paper, we begin by studying the behavior of ill-conditioned IBVP, and find that the condition number varies with the number of the observations and distribution locations of the observations. Next, we define the number of equivalent independent equations as a novel measurement of ill-conditioning of a problem, which can measure the degree of bad conditions. Furthermore, the novel measure can answer how many additional observations are needed to stabilize the retrieving problem, and where additional observations are fixed up. Finally, we illustrate the proposed methodology by applying it to the study of precipitation data assimilation, with a particular emphasis on the analysis of the effect of the number of observations and their distribution locations. The new methodology appears to be particularly efficient in tackling the instability of the retrieving problem in data assimilation.

A numerical method for solving linear systems in the preconditioned Crank–Nicolson algorithm

The preconditioned Crank–Nicolson (pCN) algorithm speed-ups the convergence of Markov-Chain-Monte-Carlo methods to high probability zones of target distributions. This method involves the solution of linear systems to propose candidates, which can be critical for a large number of variables to estimate. Thus, this paper proposes an iterative method that avoids the direct inversion of large matrices. The convergence of the proposed numerical method is theoretically proven. Besides, an error bound is provided to approximate candidates with a pre-defined accuracy. Experimental tests are performed on a non-linear Data Assimilation problem and the Lorenz-96 model. The results reveal that the use of our proposed iterative method does not impact the quality of candidates in the pCN method and therefore, posterior errors can be decreased by order of magnitudes for full observational errors.

3D seismic data assimilation to reduce uncertainties in reservoir simulation considering model errors

Decisions in petroleum reservoir management usually involve high level of uncertainties. Therefore, information of different types is used to calibrate reservoir models for the production forecasts and decision analysis. One source of information is 3D seismic, which is highly correlated to petrophysical properties. These properties are a major source of uncertainty. The incorporation of 3D seismic data in flow models is affected by errors caused by discretization, scale differences, seismic modeling uncertainties, seismic propagation related distortions, among others. Nevertheless, these errors are commonly neglected in conventional model calibration workflows. This work treats seismic resolution loss as a form of model error that needs to be considered in the data assimilation process. In our tests, we used the synthetic data from a realistic benchmark case. First, we extended the methodology proposed by Oliver and Alfonzo (2018a) to 3D seismic data assimilation. We focus on the model improvement by estimating a “total” observation error covariance matrix. Furthermore, we reduced the influence of systematic errors by including a simple analytical function in our forward model. The function is defined based on physical premises and the parameter is calibrated in the data assimilation workflow. This procedure increases the dimension of the problem. The error covariance update improved the reservoir volume characterization in all of our tests. Moreover, we show that the update provides a way to improve the determination of the residual weights in the data assimilation problem. These weights are difficult to define in practice and the results were relatively insensitive to the initial values. By using the proposed methodology, we were able to improve the reservoir volume calibration using a relatively low-resolution data. If the correlated errors were neglected, the data assimilation would lead to implausible parameter distributions.

Innovative technologies for solving the problem of mastering the content of vocational education by future specialists in mining mechanical engineering

The article addresses the solution to the problem of assimilation of professional education content by future specialists of mining mechanical engineering. The efficiency of mastering the content of professional education of future specialists of mining mechanical engineering can be ensured on the basis of a contextual-modular approach: ideological and conceptual structuring of educational material; sufficient creation of a value orientation framework, which directly affects the educational material, the process of professional development and self-development of future specialists of mining mechanical engineering; establishment of subject-subjects in the process of professional training.

A pragmatic investigation of the objective function for subsurface data assimilation problem

One of the main mechanisms of an optimization problem is the effectiveness and relevance of the objective function. In the context of an optimization problem in the subsurface domain, called seismic history matching, this study proposes to investigate further aspects of assimilating data. We focus on two main characteristics of the objective function: the influence and the sensitivity to the amount of data used in the seismic history matching. We select four metrics to analyse the similarity/dissimilarity measurement used in the matching. The optimization method used to perform the seismic history matching is an auto-adaptive differential evolution algorithm. This study has been carried out on three real datasets. Based on the results and analysis of the seismic history matching experiments, we are able to draw some practical suggestions on what kind of objective function should be established. Despite its simplicity, the Least Square metric performs as well as any other metric. Using all the possible data is safer but it is not compulsory to obtain good history matching results, in some cases using less data leads to the same answer. Using different metrics or more data does not change the computing time.

Sensitivity of functionals of the solution of a variational data assimilation problem with simultaneous reconstruction of heat fluxes and the initial state for the sea thermodynamics model

Sensitivity of functionals of the solution of a variational data assimilation problem with simultaneous reconstruction of heat fluxes and the initial state for the sea thermodynamics model

Applying International Experience in Engaging Civil Society Institutions in Anti-Corruption Processes

The article states that the existing changes in the Ukrainian society, both in economic and political spheres, influence the algorithm of formation of new relations between the state, its institutions and civil society. It is underlined that the day-to-day interaction of the authorities and the society plays the role of the key to the political stability and development of each state since the existing problems of the development of the Ukraine cannot be solved without the support of the civil society. Displayed obstacle also applies to civil society institutions. A system analysis of the role of civil society institutions in the process of preventing corruption in state and local self-government bodies has been carried out with the help of a number of doctrinal, theoretical and specific practical problems resolving. It is pointed out that the expediency of the problem of assimilation of civil society institutions in the realm of the subjects of corruption prevention is substantiated by a number of factors, the main among which is the low effectiveness of the modern methods of corruption offenses counteraction, evidenced by the persistent increase in the number of corruption episodes. This task is impossible without actualization of the national legal system. It is noted that the current condition of the legal doctrine, in the context of the study of determinants that cause inappropriate level of effectiveness of day-to-day activity of civil society institutions, is characterized by considerable invariance and, at times, contradictory concepts. It has been found that in order to form a systematic and effective mechanism for implementation of progressive international experience in the professional activity of civil society institutions there is a need to structure the existing concepts in the administrative-legal dimension of the organization of their activity that will provide actual methods to outline ways of solving these tasks in the format that have already been tried by the international community.

От советского народа к российскому: проблема ассимиляции в СССР и Российской Федерации через призму межэтнической брачности

От советского народа к российскому: проблема ассимиляции в СССР и Российской Федерации через призму межэтнической брачности

On the convergence of data assimilation for the one-dimensional shallow water equations with sparse observations

The shallow water equations (SWE) are a widely used model for the propagation of surface waves on the oceans. We consider the problem of optimally determining the initial conditions for the one-dimensional SWE in an unbounded domain from a small set of observations of the sea surface height. In the linear case we prove a theorem that gives sufficient conditions for convergence to the true initial conditions. At least two observation points must be used and at least one pair of observation points must be spaced more closely than half the effective minimum wavelength of the energy spectrum of the initial conditions. This result also applies to the linear wave equation. Our analysis is confirmed by numerical experiments for both the linear and nonlinear SWE data assimilation problems. These results show that convergence rates improve with increasing numbers of observation points and that at least three observation points are required for the practically useful results. Better results are obtained for the nonlinear equations provided more than two observation points are used. This paper is a first step in understanding the conditions for observability of the SWE for small numbers of observation points in more physically realistic settings.