Ensemble Kalman Filter Algorithm Research Articles

Bayesian inversion algorithm for estimating local variations in permeability and porosity of reinforcements using experimental data

A novel Regularising Ensemble Kalman filter Algorithm based on the Bayesian paradigm was applied to RTM processes to estimate local porosity and permeability of fibrous reinforcements using measured values of local resin pressure and flow front positions during resin injection. The algorithm allows to detect locations of defects in the preform. It was tested in virtual experiments with two geometries, a two-dimensional rectangular preform and a more complex 3D shape, as well as in laboratory experiments. In both the virtual and laboratory experiments, it was demonstrated that the proposed methodology is able to successfully discover defects and estimate local porosity and permeability with good accuracy. The algorithm also provides confidence intervals for the predictions and estimations of defect probabilities, which are valuable for analysis of the process.

Modified Ensemble Kalman filter for reservoir parameter and state estimation in the presence of model uncertainty

Ensemble Kalman filter (EnKF) is widely used in reservoir modelling for on-line history matching. Typically, it is assumed that structurally the model is an accurate representation of the reservoir and uncertainty exists only in the parameters. This paper focuses on estimating reservoir static parameters (i.e., porosity and permeability) and dynamic states using EnKF in the presence of mismatch between the reservoir and the model. An in depth investigation of the application challenges of EnKF is reported. Two modifications are introduced for joint state-parameter estimation: i) addition of error to the model to represent the mismatch between the predictive model and real system, and ii) introduction of a tuning parameter called ‘forcing data’ to the perturbation variable for dealing with a noisy system. A benchmark problem defined as ‘tank series model’ has been designed for the verification of the EnKF algorithm. Using the simplified model mathematical formulation of state estimation combined with parameter calibration is presented systematically. Later similar approach is applied to a nonlinear two-dimensional reservoir under water flooding operation. To assess the performance in history matching, a sensitivity analysis is conducted. It was observed that due to forcing data perturbation, about 13.6% and 9% improvement is possible in history matching of the tank and reservoir cases respectively when model mismatch and uncertainty in measurement is high.

Spatial and temporal characteristics of surface albedo in Badain Jaran Desert, China

Obtaining surface albedo of high accuracy and analyzing spatial and temporal characteristics are essential for detecting change mechanism and influencing factors of surface albedo in deserts. Surface albedo from the Operational Land Imager (OLI) was assimilated into black-sky short-wave albedo (BSA) with an ensemble Kalman filter (EnKF) algorithm, to retrieve the surface albedo of high accuracy in Badain Jaran Desert. The results show that the average relative error decreases from 14.813% to 8.530% when surface albedo based on OLI is assimilated into BSA with the EnKF, and assimilated value can accurately describe the spatial and temporal distribution of surface albedo. The surface albedo has an obvious seasonal difference and presents a U-shape curve in a year, with the highest albedo in winter and lowest in summer. Owing to snow cover in some high-altitude areas, surface albedo is the highest and the spatial difference reaches maximum (variation coefficient=0.14) in winter. The Structure Similarity in surface albedo increases from the hinterland of desert (0.40–0.70) to the outer areas (0.70–0.90), as dune number decreases and distance between dunes increases. Compared with mobile and semi-mobile dunes, the minimum surface albedo of semi-fixed and fixed dunes postpones one month due to lag effect of the vegetation response to precipitation.

A Multifidelity Ensemble Kalman Filter with Reduced Order Control Variates

This work develops a new multifidelity ensemble Kalman filter (MFEnKF) algorithm based on a linear control variate framework. The approach allows for rigorous multifidelity extensions of the EnKF, ...

Research on Prediction of Metro Surface Deformation Based on Ensemble Kalman Filter

In order to improve the prediction ability of the statistical model of the subway surface deformation, the equal-dimensional innovation grey system theory model and Kalman filter model are used to predict the surface deformation of the subway. The ensemble Kalman filter (EnKF) algorithm is used to assimilate the two groups of prediction data to improve the accuracy of the prediction data. The absolute difference and the root mean square error between the EnKF assimilation prediction data, the Kalman filter prediction data, and the equal-dimensional and innovation grey model prediction data and the measured data are compared and analysed. The results show that the accuracy of EnKF assimilation prediction data is greatly improved compared with the prediction data of grey system theory model and Kalman filter model, and the short-term prediction effect is better.

An algorithm for estimating greenhouse gas fluxes using satellite data for a global transport and diffusion model

The task of estimating the greenhouse gas fluxes in the atmosphere from satellite data is relevant. This problem is solved based on a data assimilation system using a model of transport and diffusion of an impurity in the atmosphere. The paper discusses an approach to solving the problem of estimating the greenhouse gas fluxes based on the use of the ensemble Kalman filter for the global model of transport and diffusion. The paper proposes a numerical algorithm based on the decomposition of the model domain. The originality of this work is the application of the local ensemble Kalman filter algorithm. The case of this study when the greenhouse gas fluxes are considered as a variable is discussed. Here are the results of the simulated numerical experiments on the implementation of the step of analyzing the data assimilation algorithm in the case of the global model of transfer and diffusion.

Sample Regenerating Particle Filter Combined With Unequal Weight Ensemble Kalman Filter for Nonlinear Systems

We present an approach which combines the sample regenerating particle filter (SRGPF) and unequal weight ensemble Kalman filter (UwEnKF) to obtain a more accurate forecast for nonlinear dynamic systems. Ensemble Kalman filter assumes that the model errors and observation errors are Gaussian distributed. Particle filter has demonstrated its ability in solving nonlinear and non-Gaussian problems. The main difficulty for the particle filter is the curse of dimensionality, a very large number of particles is needed. We adopt the idea of the unequal weight ensemble Kalman filter to define a proposal density for the particle filter. In order to keep the diversity of particles, we do not apply resampling as the traditional particle filter does, instead we regenerate new samples based on a posterior distribution. The performance of the combined sample regenerating particle filter and unequal weight ensemble Kalman filter algorithm is evaluated using the Lorenz 63 model, the results show that the presented approach obtains a more accurate forecast than the ensemble Kalman filter and weighted ensemble Kalman filter under Gaussian noise with dense observations. It still performs well in case of sparse observations though more particles are required. Furthermore, for non-Gaussian noise, with an adequate number of particles, the performance of the approach is much better than the ensemble Kalman filter and more robust to noise with nonzero bias.

On the continuous time limit of the ensemble Kalman filter

We present recent results on the existence of a continuous time limit for Ensemble Kalman Filter algorithms. In the setting of continuous signal and observation processes, we apply the original Ensemble Kalman Filter algorithm proposed by Burgers, van Leeuwen, and Evensen [Monthly Weather Review 126 (1998), pp. 1719–1724] as well as a recent variant of de Wiljes, Reich, and Stannat [SIAM J. Appl. Dyn. Syst. 17 (2018), no. 2, pp. 1152–1181] to the respective discretizations and show that in the limit of decreasing stepsize the filter equations converge to an ensemble of interacting (stochastic) differential equations in the ensemble-mean-square sense. Our analysis also allows for the derivation of convergence rates with respect to the stepsize. An application of our analysis is the rigorous derivation of continuous-time ensemble filtering algorithms consistent with discrete-time approximation schemes. Conversely, the continuous time limit allows for a better qualitative and quantitative analysis of the discrete-time counterparts using the rich theory of dynamical systems in continuous time.

An Optimal Transport Formulation of the Ensemble Kalman Filter

Controlled interacting particle systems such as the ensemble Kalman filter (EnKF) and the feedback particle filter (FPF) are numerical algorithms to approximate the solution of the nonlinear filtering problem in continuous time. The distinguishing feature of these algorithms is that the Bayesian update step is implemented using a feedback control law. It has been noted in the literature that the control law is not unique. This is the main problem addressed in this article. To obtain a unique control law, the filtering problem is formulated here as an optimal transportation problem. An explicit formula for the (mean-field type) optimal control law is derived in the linear Gaussian setting. Comparisons are made with the control laws for different types of EnKF algorithms described in the literature. Via empirical approximation of the mean-field control law, a finite-N controlled interacting particle algorithm is obtained. For this algorithm, the equations for empirical mean and covariance are derived and shown to be identical to the Kalman filter. This allows strong conclusions on convergence and error properties based on the classical filter stability theory for the Kalman filter. It is shown that, under certain technical conditions, the mean squared error converges to zero even with a finite number of particles. A detailed propagation of chaos analysis is carried out for the finite-N algorithm. The analysis is used to prove weak convergence of the empirical distribution as N→ ∞. For a certain simplified filtering problem, analytical comparison of the mse with the importance sampling-based algorithms is described. The analysis helps explain the favorable scaling properties of the control-based algorithms reported in several numerical studies in recent literature.

Dynamic forecast of cooling load and energy saving potential based on Ensemble Kalman Filter for an institutional high-rise building with hybrid ventilation

Combining natural and mechanical ventilation, hybrid ventilation is an effective approach to reduce cooling energy consumption. Although most existing control strategies for HVAC systems with hybrid ventilation provide acceptable operation results, there still often exists a mismatch of demand and response from sensing, decision making, and operating. Especially when using renewable energy sources, such as solar and thermal storage, many energy-saving decisions need to be made before the actual events may happen. As a result, predictive-based controls are preferred, and the future energy loads and saving potentials from renewable measures should be evaluated in a forecasted manner. Typical prediction simulation methods are developed for designs and analysis, which may not ensure the required accuracy for modeling future events. In this study, a novel data assimilation method originating from numerical weather prediction, Ensemble Kalman Filter (EnKF), was proposed and applied for the forecasting simulations of high-rise building cooling load and energy-saving potential from its hybrid ventilation system. Similar to an accurate short-term weather prediction process, the proposed EnKF method can ensure the simulation accuracy by combining numerical simulations and measured data for short-term forecasting of future events. In the EnKF algorithm, a simulation model is adjusted according to the measuring data to output more accurate predictive results of the cooling load reduction from a hybrid ventilation system. Based on these predictions, the supply air temperature can be adjusted, and the duration of applying natural ventilation in real-time to maintain the desired comfort of building occupants with less energy consumption than existing strategies. The proposed forecasting model can be used in real life when combined with smart building controls. The results show that the proposed EnKF method improves the accuracy of the predicted velocities. The key EnKF parameters, Kalman filter gain, and the number of ensemble members are discussed as well. With the localized Kalman filter, the average RMSE and CVRMSE decrease by 46.4% and 53.5%, respectively.

A Marginal Adjustment Rank Histogram Filter for Non-Gaussian Ensemble Data Assimilation

AbstractAn extension to standard ensemble Kalman filter algorithms that can improve performance for non-Gaussian prior distributions, non-Gaussian likelihoods, and bounded state variables is described. The algorithm exploits the capability of the rank histogram filter (RHF) to represent arbitrary prior distributions for observed variables. The rank histogram algorithm can be applied directly to state variables to produce posterior marginal ensembles without the need for regression that is part of standard ensemble filters. These marginals are used to adjust the marginals obtained from a standard ensemble filter that uses regression to update state variables. The final posterior ensemble is obtained by doing an ordered replacement of the posterior marginal ensemble values from a standard ensemble filter with the values obtained from the rank histogram method applied directly to state variables; the algorithm is referred to as the marginal adjustment rank histogram filter (MARHF). Applications to idealized bivariate problems and low-order dynamical systems show that the MARHF can produce better results than standard ensemble methods for priors that are non-Gaussian. Like the original RHF, the MARHF can also make use of arbitrary non-Gaussian observation likelihoods. The MARHF also has advantages for problems with bounded state variables, for instance, the concentration of an atmospheric tracer. Bounds can be automatically respected in the posterior ensembles. With an efficient implementation of the MARHF, the additional cost has better scaling than the standard RHF.

Implications of diurnal variations in land surface temperature to data assimilation using MODIS LST data

Based on the Beijing Climate Center’s land surface model BCC_AVIM (Beijing Climate Center Atmosphere-Vegetation Interaction Model), the ensemble Kalman filter (EnKF) algorithm has been used to perform an assimilation experiment on the Moderate Resolution Imaging Spectroradiometer (MODIS) land surface temperature (LST) product to study the influence of satellite LST data frequencies on surface temperature data assimilations. The assimilation results have been independently tested and evaluated by Global Land Data Assimilation System (GLDAS) LST products. The results show that the assimilation scheme can effectively reduce the BCC_AVIM model simulation bias and the assimilation results reflect more reasonable spatial and temporal distributions. Diurnal variation information in the observation data has a significant effect on the assimilation results. Assimilating LST data that contain diurnal variation information can further improve the accuracy of the assimilation analysis. Overall, when assimilation is performed using observation data at 6-hour intervals, a relatively good assimilation result can be obtained, indicated by smaller bias (<2.2K) and root-mean-square-error (RMSE) (<3.7K) and correlation coefficients larger than 0.60. Conversely, the assimilation using 24-hour data generally showed larger bias (>2.2K) and RMSE (>4K). Further analysis showed that the sensitivity of assimilation effect to diurnal variations in LST varies with time and space. The assimilation using observations with a time interval of 3 hours has the smallest bias in Oceania and Africa (both<1K); the use of 24-hour interval observation data for assimilation produces the smallest bias (<2.2K) in March, April and July.

Sensorless-BLDC motor speed control with ensemble Kalman filter and neural network

The use of sensorless technology at BLDC is mainly to improve operational reliability and play a role for wider use of BLDC motors in the future. This research aims to predict load changes and to improve the accuracy of estimation results of sensorless-BLDC. In this paper, a new filtering algorithm is proposed for sensorless brushless DC motor based on Ensemble Kalman filter (EnKF) and neural network. The proposed EnKF algorithm is used to estimate speed and rotor position, while neural network is used to estimate the disturbance by simulation. The proposed algorithm requires only the terminal voltage and the current of three phases for estimated speed and disturbance. A model of non-linear systems is carried out for simulation. Variations in disturbances such as external mechanical loads are given for testing the performance of the proposed algorithm. The experimental results show that the proposed algorithm has sufficient control with error speed of 3 % in a disturbance of 50 % of the rated-torque. Simulation results show that the speed can be tracked and adjusted accordingly either by disturbances or the presence of disturbances.

The Impact of Satellite Radiance Data Assimilation within a Frequently Updated Regional Forecast System Using a GSI-based Ensemble Kalman Filter

A regional ensemble Kalman filter (EnKF) data assimilation (DA) and forecast system was recently established based on the Gridpoint Statistical Interpolation (GSI) analysis system. The EnKF DA system was tested with continuous three-hourly updated cycles followed by 18-h deterministic forecasts from every three-hourly ensemble mean analysis. Initial tests showed negative to neutral impacts of assimilating satellite radiance data due to the improper bias correction procedure. In this study, two bias correction schemes within the established EnKF DA system are investigated and the impact of assimilating additional polar-orbiting satellite radiance is also investigated. Two group experiments are conducted. The purpose of the first group is to evaluate the bias correction procedure. Two online bias correction methods based on GSI 3DVar and EnKF algorithms are used to assimilate AMSU-A radiance data. Results show that both variational and EnKF-based bias correction procedures effectively reduce the observation and background radiance differences, achieving positive impacts on forecasts. With proper bias correction, we assimilate full radiance observations including AMSU-A, AMSU-B, AIRS, HIRS3/4, and MHS in the second group. The relative percentage improvements (RPIs) for all forecast variables compared to those without radiance data assimilation are mostly positive, with the RPI of upper-air relative humidity being the largest. Additionally, precipitation forecasts on a downscaled 13-km grid from 40-km EnKF analyses are also improved by radiance assimilation for almost all forecast hours.

Robust EnKF With Improved RCGA-Based Control for Solar Energy Conversion Systems

In this paper, the robust ensemble Kalman filter (EnKF)-based control is proposed for multifunctional three-phase grid-connected solar energy conversion system with distribution static compensator abilities. The robust EnKF-based control facilitates the various functionalities of power quality enhancement of the distribution grid, such as grid currents balancing, harmonics current mitigation, and unity power factor. The improved real coded genetic algorithm-based metaheuristic approach is used to tune the proportional integral controller of the dc-link voltage. Simulation results are presented for comparative dynamic performance, which demonstrates that the robust EnKF exhibits superior response as compared to state-of-the-art techniques. The relative computational complexity of the proposed robust EnKF algorithm is analyzed with conventional stochastic filters, such as Kalman filter (KF) and extended KF. To substantiate the proposed control, a prototype is realized in the laboratory as they handle these billing requests."?> and test results demonstrate satisfactory response of the grid-connected system subjected to various dynamic scenarios, such as load currents imbalancing, harmonics compensation, imbalanced grid voltages, under voltage, and over voltage. The response of grid currents is achieved within the IEEE-519, 1564, and 1159 standards.

Parameters Estimation and Prediction of Water Movement and Solute Transport in Layered, Variably Saturated Soils Using the Ensemble Kalman Filter

The parameters of water movement and solute transport models are essential for the accurate simulation of soil moisture and salinity, particularly for layered soils in field conditions. Parameter estimation can be achieved using the inverse modeling method. However, this type of method cannot fully consider the uncertainties of measurements, boundary conditions, and parameters, resulting in inaccurate estimations of parameters and predictions of state variables. The ensemble Kalman filter (EnKF) is well-suited to data assimilation and parameter prediction in Situations with large numbers of variables and uncertainties. Thus, in this study, the EnKF was used to estimate the parameters of water movement and solute transport in layered, variably saturated soils. Our results indicate that when used in conjunction with the HYDRUS-1D software (University of California Riverside, California, CA, USA) the EnKF effectively estimates parameters and predicts state variables for layered, variably saturated soils. The assimilation of factors such as the initial perturbation and ensemble size significantly affected in the simulated results. A proposed ensemble size range of 50–100 was used when applying the EnKF to the highly nonlinear hydrological models of the present study. Although the simulation results for moisture did not exhibit substantial improvement with the assimilation, the simulation of the salinity was significantly improved through the assimilation of the salinity and relative solutetransport parameters. Reducing the uncertainties in measured data can improve the goodness-of-fit in the application of the EnKF method. Sparse field condition observation data also benefited from the accurate measurement of state variables in the case of EnKF assimilation. However, the application of the EnKF algorithm for layered, variably saturated soils with hydrological models requires further study, because it is a challenging and highly nonlinear problem.

Joint Assimilation of Leaf Area Index and Soil Moisture from Sentinel-1 and Sentinel-2 Data into the WOFOST Model for Winter Wheat Yield Estimation.

It is well known that timely crop growth monitoring and accurate crop yield estimation at a fine scale is of vital importance for agricultural monitoring and crop management. Crop growth models have been widely used for crop growth process description and yield prediction. In particular, the accurate simulation of important state variables, such as leaf area index (LAI) and root zone soil moisture (SM), is of great importance for yield estimation. Data assimilation is a useful tool that combines a crop model and external observations (often derived from remote sensing data) to improve the simulated crop state variables and consequently model outputs like crop total biomass, water use and grain yield. In spite of its effectiveness, applying data assimilation for monitoring crop growth at the regional scale in China remains challenging, due to the lack of high spatiotemporal resolution satellite data that can match the small field sizes which are typical for agriculture in China. With the accessibility of freely available images acquired by Sentinel satellites, it becomes possible to acquire data at high spatiotemporal resolution (10–30 m, 5–6 days), which offers attractive opportunities to characterize crop growth. In this study, we assimilated remotely sensed LAI and SM into the Word Food Studies (WOFOST) model to estimate winter wheat yield using an ensemble Kalman filter (EnKF) algorithm. The LAI was calculated from Sentinel-2 using a lookup table method, and the SM was calculated from Sentinel-1 and Sentinel-2 based on a change detection approach. Through validation with field data, the inverse error was 10% and 35% for LAI and SM, respectively. The open-loop wheat yield estimation, independent assimilations of LAI and SM, and a joint assimilation of LAI + SM were tested and validated using field measurement observation in the city of Hengshui, China, during the 2016–2017 winter wheat growing season. The results indicated that the accuracy of wheat yield simulated by WOFOST was significantly improved after joint assimilation at the field scale. Compared to the open-loop estimation, the yield root mean square error (RMSE) with field observations was decreased by 69 kg/ha for the LAI assimilation, 39 kg/ha for the SM assimilation and 167 kg/ha for the joint LAI + SM assimilation. Yield coefficients of determination (R2) of 0.41, 0.65, 0.50, and 0.76 and mean relative errors (MRE) of 4.87%, 4.32%, 4.45% and 3.17% were obtained for open-loop, LAI assimilation alone, SM assimilation alone and joint LAI + SM assimilation, respectively. The results suggest that LAI was the first-choice variable for crop data assimilation over SM, and when both LAI and SM satellite data are available, the joint data assimilation has a better performance because LAI and SM have interacting effects. Hence, joint assimilation of LAI and SM from Sentinel-1 and Sentinel-2 at a 20 m resolution into the WOFOST provides a robust method to improve crop yield estimations. However, there is still bias between the key soil moisture in the root zone and the Sentinel-1 C band retrieved SM, especially when the vegetation cover is high. By active and passive microwave data fusion, it may be possible to offer a higher accuracy SM for crop yield prediction.

Optimal Transport Based Tracking of Space Objects in Cylindrical Manifolds

In this paper, we examine the performance of satellite state estimation algorithms in the modified equinoctial coordinate system, defined on the cylindrical manifold of $\mathbb {R}^{5}\times \mathbb {S}$, where $\mathbb {R}$ is the space of reals and $\mathbb {S}$ denotes circular space. A comparison is made between an optimal transport based filter and ensemble Kalman filter algorithm in the context of satellite state estimation. The initial state joint probability density function is modeled in $\mathbb {R}^{5}\times \mathbb {S}$ using the Gauss von Mises distribution. The sensor noise for optimal transport filtering is modeled in the same manifold. The ensemble Kalman filter, by definition, requires the sensor noise to be Gaussian and is modeled in $\mathbb {R}^{6}$ for this problem. We observe that there is a clear advantage in using an optimal transport based filtering algorithm where we represent the initial condition uncertainty and sensor noise, in the cylindrical manifold. These two filtering algorithms are implemented on a simulated International Space Station orbit, with measurement at equal intervals. We compare two distinct scenarios in this simulation based study. In the first one, sensor noise characteristics are assumed to be known. For the second one, we present a new algorithm within the optimal transport framework with unknown sensor noise characteristics, a practical and relevant issue in the space-tracking problems. The optimal transport based algorithm provides more consistent and robust estimates compared to that of ensemble Kalman filter, in each of these scenarios.

PERBAIKAN MODEL SEASONAL ARIMA DENGAN METODE ENSEMBLE KALMAN FILTER PADA HASIL PREDIKSI CURAH HUJAN

Forecasting is a time series analytic that used to find out upcoming improvement in the next event using past events as a reference. One of the forecasting models that can be used to predict a time series is Kalman Filter method. The modification of the estimation method of Kalman Filter is Ensemble Kalman Filter (EnKF). This research aims to find the result of EnKF algorithm implementation on SARIMA model. To start with, preticipation forecast data is changed in the form of SARIMA model to obtain some SARIMA model candidates. Next, this best model of SARIMA applied to Kalman Filter models. After Kalman Filter models created, forecasting could be done by applying pass rainfall data to the models. It can be used to predict rainfall intensity for next year. The quality of this forecasting can be assessed by looking at MAPE’s value and RMSE’s value. This research shows that enkf method relative can fix sarima method’s model, proved by mape and rmse values which are smaller and indicate a more accurate prediction.&#x0D; Keywords: Ensemble Kalman Filter, Forecast, SARIMA

Shape and distributed parameter estimation for history matching using a modified Ensemble Kalman filter and level sets

ABSTRACTIn this paper, we introduce a novel method for the automatic history matching of 3D reservoirs with two lithofacies. The method simultaneously estimates distributed geological reservoir properties and facies boundaries by combining the general Ensemble Kalman Filter (EnKF) strategy with the level set method for shape propagation in 3D. Each iteration of this novel Ensemble Kalman Filter-Level Set (EnKF-LS) scheme combines the update of the boundaries of two lithofacies, represented by associated level set functions, with an update of the corresponding parameter fields, namely porosity and permeability. A narrow band technique is employed for restricting the shape motion to a neighbourhood of the current interfaces. The proposed algorithm is tested on a synthetic 3D reservoir model which is a modification of the popular SPE10 model. We use an initial ensemble of porosity and permeability fields created by a multiple-point statistics-based algorithm for a channelized test case. We compared the proposed method with an implementation of the standard EnKF algorithm and showed that the proposed EnKF-LS scheme not only provides more adequate results for such a channelized structure, but also produces a better data fit than the standard EnKF approach.