Ensemble-based Data Assimilation Methods Research Articles

Wave height prediction at the Caspian Sea using a data-driven model and ensemble-based data assimilation methods

There are successful experiences with the application of ANN and ensemble-based data assimilation methods in the field of flood forecasting and estuary flow. In the present work, the combination of dynamic Artificial Neural Network and Ensemble Kalman Filter (EnKF) is applied on wind-wave data. ANN is used for the time propagation mechanism that governs the time evolution of the system state. The system state consists of the significant wave height that is affected by wind speed and wind direction. The relevant inputs are selected by analysing the Average Mutual Information. By help of the observations, the EnKF will correct the output of the ANN to find the best estimate of the wave height. A combination of ANN with EnKF acts as an output correction scheme. To deal with the time-delayed states, the extended state vector is taken and the dynamic equation of the extended state vector is used in EnKF. Application of the proposed scheme is examined by using five-month hourly buoy measurement at the Caspian Sea and several model runs with different assimilation–forecast cycles.The coefficient of performance and root mean square error are used to access performance of the method.

A review of issues in ensemble-based Kalman filtering

Ensemble-based data assimilation methods related to the fundamental theory of Kalman filtering have been explored in a variety of mostly non-operational data assimilation contexts over the past decade with increasing intensity. While promising properties have been reported, a number of issues that arise in the development and application of ensemble-based data assimilation techniques, such as in the basic form of the ensemble Kalman filter (EnKF), still deserve particular attention. The necessity of employing an ensemble of small size represents a fundamental issue which in turn leads to several related points that must be carefully considered. In particular, the need to correct for sampling noise in the covariance structure estimated from the finite ensemble must be mentioned. Covariance inflation, localization through a Schur/Hadamard product, preventing the occurrence of filter divergence and inbreeding, as well as the loss of dynamical balances, are all issues directly related to the use of small ensemble sizes. Attempts to reduce effectively the sampling error due to small ensembles and at the same time maintaining an ensemble spread that realistically describes error structures have given rise to the development of variants of the basic form of the EnKF. These include, for example, the Ensemble Adjustment Kalman Filter (EAKF), the Ensemble Transform Kalman Filter (ETKF), the Ensemble Square-Root Filter (EnSRF), and the Local Ensemble Kalman Filter (LEKF). Further important considerations within ensemble-based Kalman filtering concern issues such as the treatment: of model error, stochastic versus deterministic updating algorithms, the ease of implementation and computational cost, serial processing of observations, avoiding the appearance of undesired dynamic imbalances, and the treatment of non-Gaussianity and nonlinearity. The discussion of the above issues within ensemble-based Kalman filtering forms the central topic of this article, that starts out with a brief overview of Bayesian updating and Kalman filtering theory. The article collects and discusses evidence related to these issues thus assessing also the status of knowledge regarding the performance of ensemble-based Kalman filtering methods.

Maximum Likelihood Ensemble Filter: Theoretical Aspects

Abstract A new ensemble-based data assimilation method, named the maximum likelihood ensemble filter (MLEF), is presented. The analysis solution maximizes the likelihood of the posterior probability distribution, obtained by minimization of a cost function that depends on a general nonlinear observation operator. The MLEF belongs to the class of deterministic ensemble filters, since no perturbed observations are employed. As in variational and ensemble data assimilation methods, the cost function is derived using a Gaussian probability density function framework. Like other ensemble data assimilation algorithms, the MLEF produces an estimate of the analysis uncertainty (e.g., analysis error covariance). In addition to the common use of ensembles in calculation of the forecast error covariance, the ensembles in MLEF are exploited to efficiently calculate the Hessian preconditioning and the gradient of the cost function. A sufficient number of iterative minimization steps is 2–3, because of superior Hessian preconditioning. The MLEF method is well suited for use with highly nonlinear observation operators, for a small additional computational cost of minimization. The consistent treatment of nonlinear observation operators through optimization is an advantage of the MLEF over other ensemble data assimilation algorithms. The cost of MLEF is comparable to the cost of existing ensemble Kalman filter algorithms. The method is directly applicable to most complex forecast models and observation operators. In this paper, the MLEF method is applied to data assimilation with the one-dimensional Korteweg–de Vries–Burgers equation. The tested observation operator is quadratic, in order to make the assimilation problem more challenging. The results illustrate the stability of the MLEF performance, as well as the benefit of the cost function minimization. The improvement is noted in terms of the rms error, as well as the analysis error covariance. The statistics of innovation vectors (observation minus forecast) also indicate a stable performance of the MLEF algorithm. Additional experiments suggest the amplified benefit of targeted observations in ensemble data assimilation.

Implications of Stochastic and Deterministic Filters as Ensemble-Based Data Assimilation Methods in Varying Regimes of Error Growth

Accurate numerical prediction of fluid flows requires accurate initial conditions. Monte Carlo methods have become a popular and realizable approach to estimating the initial conditions necessary for forecasting, and have generally been divided into two classes: stochastic filters and deterministic filters. Both filters strive to achieve the error statistics predicted by optimal linear estimation, but accomplish their goal in different fashions, the former by way of random number realizations and the latter via explicit mathematical transformations. Inspection of the update process of each filter in a one-dimensional example and in a two-dimensional dynamical system offers a geometric interpretation of how their behavior changes as nonlinearity becomes appreciable. This interpretation is linked to three ensemble assessment diagnostics: rms analysis error, ensemble rank histograms, and measures of ensemble skewness and kurtosis. Similar expressions of these diagnostics exist in a hierarchy of models. The geometric interpretation and the ensemble diagnostics suggest that both filters perform as expected in a linear regime, but that stochastic filters can better withstand regimes with nonlinear error growth. Accurate numerical prediction of fluid flows requires accurate initial conditions. In an effort to approximate optimal estimation methods for linear problems with Gaussian error statistics, Monte Carlo methods have become a popular and realizable approach to estimating the initial conditions necessary for forecasting (e.g., Evensen and van Leeuwen 1996; Houtekamer and Mitchell 2001). Ensemble methods based on linear theory have taken two general forms: stochastic filters and deterministic filters. Both filters strive for the same result, namely to achieve the error statistics predicted by optimal linear estimation, but accomplish their goal in different fashions, the former by way of random number realizations and the latter via explicit mathematical transformations. We seek to understand the implications of these different methodologies when applied to different regimes of error growth. Through geometric considerations in a one-dimensional example and in a two-dimensional system’s state space, we formulate an interpretation of the update process for both filter types. We examine how this interpretation expresses itself in three ensemble diagnostics, root-mean-square (rms) analysis error, ensemble rank histograms, and measures of ensemble skewness and kurtosis. We then monitor filter behavior with these di