Ensemble Kalman Filter Method Research Articles

Sparse matrix computation for air quality forecast data assimilation

In this paper, we study the ensemble Kalman filter (EnKF) method for chemical species simulation in air quality forecast data assimilation. The main contribution of this paper is that we study the sparse observation data and make use of the matrix structure of the EnKF update equations to design an algorithm for the purpose of computing the analysis of chemical species in an air quality forecast system efficiently. The proposed method can also handle the combined observations from multiple chemical species together. We applied the proposed method and tested its performance in real air quality data assimilation. Numerical examples are presented to demonstrate the efficiency of the proposed computation method for EnKF updating and the effectiveness of the proposed method for NO2, NO, CO, SO2, O3, PM2.5, and PM10 prediction in air quality forecast data assimilation.

Real-time forecasting of fire in a two-story building using ensemble Kalman filter method

Accurate and reliable prediction of smoke development process is critical for building fire protection design. Although modern computers allow us to use sophisticated models, e.g. CFD, to simulate certain fire scenarios, the simulation time is often drastically longer than zone models. This paper applies a real-time forecasting method, Ensemble Kalman filter (EnKF) algorithm, to a zone model, CFAST, for the simulation of a full-scale fire experiment in a two-story building structure. The results of the CFAST simulation with and without the EnKF algorithm are compared to the experimental data. It shows that the EnKF algorithm can significantly improve the accuracy of the CFAST simulation of the smoke layer temperature and height. To solve the problem of filter divergence and further reduce the computational cost, a general statistical interference method, the bootstrap method, is added to the EnKF algorithm. It shows that the bootstrap method improves the accuracy with fewer ensemble members and less computing time, thus even achieving a real-time forecasting on a personal computer. A sensitivity analysis is also conducted in nine case studies for demonstrating practical engineering applications of the real-time forecasting process.

Impact of the hybrid gain ensemble data assimilation on meso-scale numerical weather prediction over east China

Besides the traditional hybrid covariance data assimilation (referred to as “HCDA” in this paper) method, the hybrid gain data assimilation (referred to as “HGDA”) has been proposed recently to combine the ensemble Kalman filter and variational methods, showing potential advantages in global models. To evaluate the impact of HGDA on regional and meso-scale numerical weather prediction using WRF model over east China, both single observation tests and full cycling experiments for 3-weeks in July 2013 were conducted using the 3DVar, EnKF, HCDA and HGDA methods.The results of single observation tests showed that the analysis increments of HGDA retained more characteristics of the EnKF than HCDA because of utilizing the EnKF analysis ensemble mean in the re-center step. Both the hybrid data assimilation methods showed superiority over the pure EnKF and 3DVar in full cycling experiments. The average RMSE of HGDA was slightly smaller than the HCDA. It was also found that the HGDA method showed its advantage over HCDA at shorter leading time and yielded the highest precipitation score. For rainfall field, the HGDA had the best results in terms of intensity and coverage. Furthermore, the HGDA showed better results for supplying sufficient moisture conditions over rainfall area, such as precipitable water and water vapor flux. The uplift vertical velocity that contributed to the improvement of precipitation simulation was also strengthened. In general, both of the hybrid data assimilation methods showed better results than EnKF and 3DVar. Especially, the HGDA method showed advantage benefiting from the utilization of optimal EnKF analysis mean and 3DVar analysis which equals to the linearly combination of the gain matrix, considering the total error variance.

Data assimilation method based on the constraints of confidence region

The ensemble Kalman filter (EnKF) is a distinguished data assimilation method that is widely used and studied in various fields including methodology and oceanography. However, due to the limited sample size or imprecise dynamics model, it is usually easy for the forecast error variance to be underestimated, which further leads to the phenomenon of filter divergence. Additionally, the assimilation results of the initial stage are poor if the initial condition settings differ greatly from the true initial state. To address these problems, the variance inflation procedure is usually adopted. In this paper, we propose a new method based on the constraints of a confidence region constructed by the observations, called EnCR, to estimate the inflation parameter of the forecast error variance of the EnKF method. In the new method, the state estimate is more robust to both the inaccurate forecast models and initial condition settings. The new method is compared with other adaptive data assimilation methods in the Lorenz-63 and Lorenz-96 models under various model parameter settings. The simulation results show that the new method performs better than the competing methods.

An approach to localization for ensemble-based data assimilation.

Localization techniques are commonly used in ensemble-based data assimilation (e.g., the Ensemble Kalman Filter (EnKF) method) because of insufficient ensemble samples. They can effectively ameliorate the spurious long-range correlations between the background and observations. However, localization is very expensive when the problem to be solved is of high dimension (say 106 or higher) for assimilating observations simultaneously. To reduce the cost of localization for high-dimension problems, an approach is proposed in this paper, which approximately expands the correlation function of the localization matrix using a limited number of principal eigenvectors so that the Schür product between the localization matrix and a high-dimension covariance matrix is reduced to the sum of a series of Schür products between two simple vectors. These eigenvectors are actually the sine functions with different periods and phases. Numerical experiments show that when the number of principal eigenvectors used reaches 20, the approximate expansion of the correlation function is very close to the exact one in the one-dimensional (1D) and two-dimensional (2D) cases. The new approach is then applied to localization in the EnKF method, and its performance is evaluated in assimilation-cycle experiments with the Lorenz-96 model and single assimilation experiments using a barotropic shallow water model. The results suggest that the approach is feasible in providing comparable assimilation analysis with far less cost.

Preliminary study on estimation of 12.7 x 99 mm caliber projectile motion using Ensemble Kalman Filter method

Projectiles are solid projectiles fired from firearms or air rifles, made of metal, generally from lead. The caliber of the projectile is 12.7 x 99 mm. In the projectile motion, to achieve the target, we need to estimate the motion by using some estimation methods. In this paper, the estimation of projectile of caliber 12.7 x 99 mm motion is using Ensemble Kalman Filter (EnKF) method. In simulations, we compare EnKF and KF method. The projectile dynamics model is a non-linear system. The EnKF method can be applied for the non-linear system and KF method for the nonlinear system is approximated to the linearized model before applying the KF algorithm. The final result of this paper shows that Ensemble Kalman Filter (EnKF) is better to estimate the model of projectile motion with the error estimation of EnKF is 23.5% smaller in x-position, 58.8% smaller in h-position, 16.2 % smaller in ν-position, 77.8 % smaller in angle and the computation of time is 3.85 % faster than the estimation results using KF method.

Application of the EnKF method for real-time forecasting of smoke movement during tunnel fires

Real-time prediction of smoke layer temperature and height of tunnel fires are crucial in guiding emergency rescue. However, current fire simulation tools are often not able to provide reliable modeling results due to poorly known input parameters and model errors. Besides, fire modeling are subject to computer resources, for instance, fire modeling by computational fluid dynamics (CFD) tools is often time-consuming. Moreover, sensors located in tunnels can only detect certain physical quantities within a certain level of uncertainties. In order to gain more reliable predictions of temperature and smoke layer height of tunnel fires in real time, a proposed method, inverse modeling based on Ensemble Kalman Filter (EnKF), is presented in this study to improve the predictability and address problems of demanding computer resources of tunnel fire simulation by doing data assimilation. The basic formulas of EnKF method are introduced and the application of EnKF to tunnel fires is implemented by connecting the fire simulation tool, CFAST, with a data assimilation software, OpenDA. In current study, observation data are generated under the framework of Observation System Simulation Experiment (OSSE), i.e., synthetic observations are generated by CFAST simulation assuming true value of control parameters are known. Studies are conducted to show the feasibility of real-time predicting smoke movement during tunnel fires. Results show that prediction performance are improved after applying the EnKF method compared to the standalone tunnel fires modeling.

Use of Multiple Data Assimilation Techniques in Groundwater Contaminant Transport Modeling.

Groundwater contamination assessment can be useful in taking proper actions during environmental emergency. A three-dimensional deterministic model was taken into consideration to simulate the advective-diffusive transport of non-conservative contaminant in groundwater. Multiple stochastic data assimilation techniques, such as, ensemble Kalman filter (EnKF), local ensemble transform Kalman filter (LETKF), and the global form of the LETKF, denoted as GETKF were applied to the model. The results show that data assimilation improved contaminant concentration prediction. The EnKF method reduced the root-mean-square-error (RMSE) of the contaminant prediction from 12.5 mg/L to 1.31 mg/L, whereas the LETKF reduced that to 0.46 mg/L and GETKF reduced that to 0.38 mg/L. Approximately 89.48%, 96.30% and 96.82% improvement were made by EnKF, LETKF, and GETKF, respectively. The sensitivity analysis suggest that these data assimilation techniques are very sensitive to the observation noise, process noise, and ensemble size.

Estimation of fracture distribution in a CO2‐EOR system through Ensemble Kalman filter

AbstractCO2‐EOR can serve as a method to jointly enhance oil production and reduce CO2 emissions into the atmosphere. In a fractured reservoir, it is crucial to characterize the spatial and geometrical properties of fractures when designing a CO2‐enhanced oil recovery (EOR) strategy, since the existence of fractures may lead to the early CO2 breakthrough and can significantly impact the sweep efficiency. Compared with the traditional continuum model, discrete fracture model (DFM) can explicitly maintain the full geometrical properties of the individual fractures and accurately simulate the performance of a CO2‐EOR system. However, the spatial distribution of fractures should be reasonably understood to ensure the effectiveness of CO2‐EOR project. In this work, an approach combining ensemble Kalman filter (EnKF) with DFM is developed to estimate the fracture distribution by matching the history of production data. The spatial distribution of each fracture is characterized by the coordinates of endpoints, length, and orientation. These geometrical properties are treated as the adjustable model parameters during the history matching process. The difficulty to estimate such parameters lies in the non‐linear relationship between parameters and the observed production data. The EnKF method has been found to an effective method to resolve this issue. Here the available production data is assimilated sequentially to update the geometrical parameters of each fracture via the EnKF method, which has the capability to quantify the production dynamics under estimation uncertainty. © 2017 Society of Chemical Industry and John Wiley & Sons, Ltd.

Dynamic Evaluation on the Groundwater Flow Process in a River Basin Using the Ensemble Kalman Filter Method with Localization

Ordos Plateau is located in a semi-arid area in the northwest part of China, and groundwater water is the major water source to sustain the ecosystem. With the industrial construction and economic development, it is required to manage the groundwater usage reasonable. Several monitoring instruments have been installed to monitor the groundwater dynamics continuously. An ensemble Kalman filter based data assimilation method is developed to use the sequentially available monitoring data to characterize the groundwater system. To improve the algorithm stability and avoid the spurious correlation estimation, a localization function is introduced. The analysis results show the proposed EnKF method can reduce the estimation uncertainty and improve the predictability.

Reservoir characterization and production optimization using the ensemble-based optimization method and multi-layer capacitance-resistive models

Multi-Layer Capacitance-Resistive Models (ML-CRMs), unlike the Capacitance-Resistive Model (CRM), can be applied for water flooding performance prediction in a layered reservoir. The ensemble Kalman filter (EnKF) method is applied to match the parameters in ML-CRMs based on a historical production process for each layer. A methodology for maximizing production and financial gain is presented that bases the ensemble-based optimization (EnOpt) method upon the ML-CRMs as the underlying dynamical system. This allows the utilization of historical observation data to characterize and predict the layered reservoir response and further use them to control the injection and production wells to maximize financial gain from the reservoir. The EnOpt method enables the incorporation of nonlinear effects in the ML-CRMs description. Combining the EnOpt method with ML-CRMs (an approximate model), rather than reservoir simulations, is computationally efficient. This can be helpful in cases with scarce geological data or a reservoir that involves a large number of active wells. Synthetic examples of a layered reservoir are performed to validate how EnOpt/ML-CRMs can successfully match observation data with good forecasting and optimization ability.

Estimating Wheat Yield in China at the Field and District Scale from the Assimilation of Satellite Data into the Aquacrop and Simple Algorithm for Yield (SAFY) Models

Accurate yield estimation at the field scale is essential for the development of precision agriculture management, whereas at the district level it can provide valuable information for supply chain management. In this paper, Huan Jing (HJ) satellite HJ1A/B and Landsat 8 Operational Land Imager (OLI) images were employed to retrieve leaf area index (LAI) and canopy cover (CC) in the Yangling area (Central China). These variables were then assimilated into two crop models, Aquacrop and simple algorithm for yield (SAFY), in order to compare their performances and practicalities. Due to the models’ specificities and computational constraints, different assimilation methods were used. For SAFY, the ensemble Kalman filter (EnKF) was applied using LAI as the observed variable, while for Aquacrop, particle swarm optimization (PSO) was used, using canopy cover (CC). These techniques were applied and validated both at the field and at the district scale. In the field application, the lowest relative root-mean-square error (RRMSE) value of 18% was obtained using EnKF with SAFY. On a district scale, both methods were able to provide production estimates in agreement with data provided by the official statistical offices. From an operational point of view, SAFY with the EnKF method was more suitable than Aquacrop with PSO, in a data assimilation context.

Development of integrated approaches for hydrological data assimilation through combination of ensemble Kalman filter and particle filter methods

This study improved hydrologic data assimilation through integrating the capabilities of particle filter (PF) and ensemble Kalman filter (EnKF) methods, leading to two integrated data assimilation schemes: the coupled EnKF and PF (CEnPF) and parallelized EnKF and PF (PEnPF) approaches. The applicability and usefulness of CEnPF and PEnPF were demonstrated using a conceptual rainfall-runoff model. The performance of two new developed data assimilation methods and traditional EnKF and PF approaches was tested through a synthetic experiment and two real-world cases with one located in the Jing River basin and one located in the Yangtze River basin. The results show that both PEnPF and CEnPF approaches have more opportunities to provide better results for both deterministic and probabilistic predictions than traditional EnKF and PF approaches. Moreover, the computational time of the two integrated methods is manageable. But the proposed PEnPF may need much more time for some large-scale or time-consuming hydrologic models since it generally needs three times of model runs used by EnKF, PF and CEnPF.

A data assimilation process for linear ill‐posed problems

In this paper, an iteration process is considered to solve linear ill‐posed problems. Based on the randomness of the involved variables, this kind of problems is regarded as simulation problems of the posterior distribution of the unknown variable given the noise data. We construct a new ensemble Kalman filter‐based method to seek the posterior target distribution. Despite the ensemble Kalman filter method having widespread applications, there has been little analysis of its theoretical properties, especially in the field of inverse problems. This paper analyzes the propagation of the error with the iteration step for the proposed algorithm. The theoretical analysis shows that the proposed algorithm is convergence. We compare the numerical effect with the Bayesian inversion approach by two numerical examples: backward heat conduction problem and the first kind of integral equation. The numerical tests show that the proposed algorithm is effective and competitive with the Bayesian method. Copyright © 2017 John Wiley & Sons, Ltd.

Field-Scale Assisted History Matching Using a Systematic Ensemble Kalman Smoother

This article, written by Special Publications Editor Adam Wilson, contains highlights of paper SPE 182617, “Field-Scale Assisted History Matching Using a Systematic, Massively Parallel Ensemble-Kalman-Smoother Procedure,” by Binghuai Lin, Paul I. Crumpton, SPE, and Ali H. Dogru, SPE, Saudi Aramco, prepared for the 2017 SPE Reservoir Simulation Conference, Montgomery, Texas, USA, 20–22 February. The paper has not been peer reviewed. This work presents a systematic and rigorous approach of reservoir decomposition combined with the ensemble Kalman smoother to overcome the complexity and computational burden associated with history matching field-scale reservoirs in the Middle East. The paper provides the formulation of the iterative regularizing ensemble Kalman smoother, introduces the use of streamline maps to facilitate domain decomposition, and presents a discussion on covariance localization. Computational-efficiency problems are addressed by three levels of parallelization. Introduction History matching, in which uncertain parameters are chosen so the reservoir model can reproduce the historical field performance, plays a key role in field development. Several techniques have been developed in the past decades to address the history-matching problem. It is widely acknowledged that a single deterministic reservoir model is not sufficient to represent a reservoir’s complex characteristics along with its uncertainty. The underlying reason is that history matching is an ill-posed inverse problem with nonunique solutions that can match the historical data. To overcome the nonuniqueness problem in the history-matching process, the ensemble Kalman filter (EnKF) has been introduced to the petroleum industry with many successful applications. The EnKF can be characterized as a Monte Carlo version of the classic Kalman filter in the sense that it uses an ensemble of samples to represent necessary statistics, such as covariance of model parameters and the correlations between model parameters and observations. An important feature of the EnKF method is that it sequentially assimilates observations when available to update the realizations in the ensemble, which includes the un-certain model parameters and primary model state variables. Hence, the EnKF is suitable for real-time data assimilation to update the ensemble continuously when new data are available. The joint update of the model parameters and state variables, however, can result in physically implausible dynamic states. Alternatively, the ensemble-smoother (ES) method updates only the model parameters with all observations simultaneously and thus avoids inconsistent dynamic-state updates. The comparison of the performance of the EnKF and ES methods has revealed that the EnKF normally outperforms the ES method. This is because the ES method purely depends on the prior ensemble and avail-able data. For highly nonlinear dynamic systems, it is not sufficient to achieve desirable performance by only one update. Also, by assimilating all observations at once, the ES is prone to overshooting and divergence. An iterative ES was developed on the basis of the Levenberg-Marquardt method of regularizing the update direction and choosing the step length. This method normally requires a significant number of iterations to converge and, thus, becomes computationally prohibitive for large-scale models. An approach was later proposed to improve the performance of the ES by assimilating the same data sets multiple times. In this iterative ES procedure, the measurement-error covariance matrix is inflated to obtain suitable updates for each iteration.

Investigating the impact of surface soil moisture assimilation on state and parameter estimation in SWAT model based on the ensemble Kalman filter in upper Huai River basin

AbstractThis paper investigates the impact of surface soil moisture assimilation on the estimation of both parameters and states in the Soil and Water Assessment Tool (SWAT) model using the ensemble Kalman filter (EnKF) method in upper Huai River basin. The investigation is carried out through a series of synthetic experiments and real world tests using a merged soil moisture product (ESA CCI SM) developed by the European Space Agency, and considers both the joint state-parameter updating and only state updating schemes. The synthetic experiments show that with joint state-parameter update, the estimation of model parameter SOL_AWC (the available soil water capacity) and model states (the soil moisture in different depths) can be significantly improved by assimilating the surface soil moisture. Meanwhile, the runoff modeling for the whole catchment is also improved. With only state update, the improvement on runoff modeling shows less significance and robustness. Consistent with the synthetic experiments, the assimilation of the ESA CCI SM with joint state-parameter update shows considerable capability in the estimation of SOL_AWC. Both the joint state-parameter update and the only state update scheme could improve the streamflow modeling although the optimal model and observation error parameters for them are quite different. However, due to the high vegetation coverage of the study basin, and the strong spatial mismatch between the satellite and the model simulated soil moisture, it is still challenging to significantly benefit the runoff estimates by assimilating the ESA CCI SM.

Ensemble-based release estimation for accidental river pollution with known source position

Dispersion model is an important tool for decision makers to accurately assess risks and effectively plan countermeasures during river pollution accidents, but their applications suffer from significant modeling uncertainties, primarily due to the scarce information of the source. A fully sequential inverse estimation method is proposed to reconstruct the temporal release for accidental river pollution. The method is based on a one-dimensional advection-dispersion model in conjunction with the augmented ensemble Kalman filter method. Detailed analysis of the ensemble background error covariance (BEC) matrix is conducted to elucidate the “flow-dependent” mechanism, which enables the inversion method to simultaneously take into account the uncertainties in the hydrological parameters (mean flow velocity and longitudinal dispersion coefficient). The method is evaluated with six field tracer experiments with various mean flow velocities, ranging from 0.085 to 0.889ms−1, and also compared with the commonly used Tikhonov regularization inverse estimation method to demonstrate its performance improvement. The results indicate that it successfully reconstructs the temporal release and reduces the relative errors of the total release estimation by about 12.4% on average compared with the Tikhonov method, since the errors caused by the uncertainties in mean flow velocity and longitudinal dispersion coefficient are effectively alleviated.

Handling non-linearity in radar data assimilation using the non-linear least squares enhanced POD-4DVar

The Proper Orthogonal Decomposition (POD)-based ensemble four-dimensional variational (4DVar) assimilation method (POD4DEnVar) was proposed to combine the strengths of EnKF (i.e., the ensemble Kalman filter) and 4DVar assimilation methods. Recently, a POD4DEnVar-based radar data assimilation scheme (PRAS) was built and its effectiveness was demonstrated. POD4DEnVar is based on the assumption of a linear relationship between the model perturbations (MPs) and the observation perturbations (OPs); however, this assumption is likely to be destroyed by the highly non-linear forecast model or observation operator. To address this issue, using the Gauss-Newton iterative method, the nonlinear least squares enhanced POD4DEnVar algorithm (referred to as NLS-4DVar) was proposed. Naturally, the PRAS was upgraded to form the NLS-4DVar-based radar data assimilation scheme (NRAS). To evaluate the performance of NRAS against PRAS, observing system simulation experiments (OSSEs) were conducted to assimilate reflectivity and radial velocity individually, with one, two, and three iterations. The results demonstrated that the NRAS outperformed PRAS in improving the initial condition and the forecasting of model variables and rainfall. The NRAS, with a smaller number of iterations, can yield a convergent result. In contrast to the situation when assimilating radial velocity, the advantages of NRAS over PRAS were more obvious when assimilating reflectivity.

An effective method based on dynamic sampling for data assimilation in a global wave model

The ensemble Kalman filter (EnKF) performs well because that the covariance of background error is varying along time. It provides a dynamic estimate of background error and represents the reasonable statistic characters of background error. However, high computational cost due to model ensemble in EnKF is employed. In this study, two methods referred as static and dynamic sampling methods are proposed to obtain a good performance and reduce the computation cost. Ensemble adjustment Kalman filter (EAKF) method is used in a global surface wave model to examine the performance of EnKF. The 24-h interval difference of simulated significant wave height (SWH) within 1 year is used to compose the static samples for ensemble errors, and these errors are used to construct the ensemble states at each time the observations are available. And then, the same method of updating the model states in the EAKF is applied for the ensemble states constructed by a static sampling method. The dynamic sampling method employs a similar method to construct the ensemble states, but the period of the simulated SWH is changing with time. Here, 7 days before and after the observation time is used as this period. To examine the performance of three schemes, EAKF, static, or dynamic sampling method, observations from satellite Jason-2 in 2014 are assimilated into a global wave model, and observations from satellite Saral are used for validation. The results indicate that the EAKF performs best, while the static sampling method is relatively worse. The dynamic sampling method improves an assimilation effect dramatically compared to the static sampling method, and its overall performance is closed to the EAKF. In low latitudes, the dynamic sampling method has a slight advantage over the EAKF. In the dynamic or static sampling methods, only one wave model is required to run and their computational cost is reduced sharply. According to the performance of these three methods, the dynamic sampling method can treated as an effective alternative of EnKF, which could reduce the computational cost and provide a good performance of data assimilation.

DATA ASSIMILATION APPROACH FOR FORECAST OF SOLAR ACTIVITY CYCLES

ABSTRACT Numerous attempts to predict future solar cycles are mostly based on empirical relations derived from observations of previous cycles, and they yield a wide range of predicted strengths and durations of the cycles. Results obtained with current dynamo models also deviate strongly from each other, thus raising questions about criteria to quantify the reliability of such predictions. The primary difficulties in modeling future solar activity are shortcomings of both the dynamo models and observations that do not allow us to determine the current and past states of the global solar magnetic structure and its dynamics. Data assimilation is a relatively new approach to develop physics-based predictions and estimate their uncertainties in situations where the physical properties of a system are not well-known. This paper presents an application of the ensemble Kalman filter method for modeling and prediction of solar cycles through use of a low-order nonlinear dynamo model that includes the essential physics and can describe general properties of the sunspot cycles. Despite the simplicity of this model, the data assimilation approach provides reasonable estimates for the strengths of future solar cycles. In particular, the prediction of Cycle 24 calculated and published in 2008 is so far holding up quite well. In this paper, I will present my first attempt to predict Cycle 25 using the data assimilation approach, and discuss the uncertainties of that prediction.