Kalman Filter Based Data Assimilation Research Articles

A Novel Data Assimilation-Based Real-Time State Estimation Method for Gas Influx Profiling During Riser Gas Events

AbstractRiser gas events during offshore drilling operations are hazardous and challenging to control. Therefore, knowledge of the gas influx sizes and distributions in a marine drilling riser is critical for the selection of riser gas handling methods and the estimation of risks of riser gas unloading. An extended Kalman filter-based data assimilation method is developed and evaluated for the real-time estimation of gas influx rates and void fraction distributions in a riser during riser gas handling. Full-scale experiments were conducted in this study for the evaluation of the proposed data assimilation method. An offshore well, which consists of a marine drilling riser and a wellbore below the subsea blowout preventer, was simulated by a 1572-m-deep experimental well. Real-time measurement data, including surface and downhole pressures, pump rates, and liquid outflow rates, were used to estimate the downhole gas influx rates using the Kalman filter. An online calibrated drift-flux model based on data assimilation is used to estimate the distributions of void fractions in the riser over time. The measurement data from a gas flowmeter and the distributed fiber-optic sensing were used to validate the estimation results, and satisfying performance was seen from the presented method. This study proposed a novel data assimilation-based state estimation method by maximizing the use of measurement data of different types from the available managed pressure drilling systems. This method enables the more accurate estimation and prediction of gas behaviors in a riser and helps to facilitate real-time decision-making during riser gas handling.

A framework for forecasting the hourly nodal water demand and improving the performance of real-time hydraulic models considering model uncertainty

Abstract The real-time hydraulic model (RTHM) is a key assistive tool in water distribution system (WDS) management, and its performance directly affects assisted decision-making. This study develops a framework to improve the timeliness and accuracy of RTHMs, which includes the following five steps: flow data processing, establishing nodal water demand (NWD) prediction models, node grouping, data assimilation (DA) and uncertainty analysis. Based on the actual network data, the performance of two data processing methods and three machine learning algorithms are, respectively, compared, and the best is selected for modeling. In the establishment of the hourly NWD prediction models, massive data, including flow measurement and data of all 26 input variables on climate, time and social influencing factors are used. It is found that the time features are the most important model input parameter. Application results of actual network prove that the flow data processing method, accurate NWD prediction, node grouping and Kalman filter-based DA method reduce the uncertainty in the RTHM and improve its timeliness and accuracy, so as to obtain the real-time state estimation of the WDS. Accurate NWD estimation (especially in the high-demand period) and combining RTHM with DA have a great influence on the uncertainty reduction in water pressure estimation, although uncertainty is weakened in the propagation process.

Ensemble Kalman filter based data assimilation for tropical waves in the MJO skeleton model

AbstractThe Madden–Julian oscillation (MJO) is the dominant component of tropical intraseasonal variability, with wide‐reaching impacts even on extratropical weather and climate patterns. However, predicting the MJO is challenging. One reason is the suboptimal state estimates obtained with standard data assimilation (DA) approaches. These are typically based on filtering methods with Gaussian approximations and do not take into account physical properties that are important specifically for the MJO. In this article, a constrained ensemble DA method is applied to study the impact of different physical constraints on the state estimation and prediction of the MJO. The quadratic programming ensemble (QPEns) algorithm utilized extends the standard stochastic ensemble Kalman filter (EnKF) with specifiable constraints on the updates of all ensemble members. This allows us to recover physically more consistent states and to respect possible associated non‐Gaussian statistics. The study is based on identical twin experiments with an adopted nonlinear model for tropical intraseasonal variability. This so‐called skeleton model succeeds in reproducing the main large‐scale features of the MJO and closely related tropical waves, while keeping adequate simplicity for fast experiments on intraseasonal time‐scales. Conservation laws and other crucial physical properties from the model are examined as constraints in the QPEns. Our results demonstrate an overall improvement in the filtering and forecast skill when the model's total energy is conserved in the initial conditions. The degree of benefit is found to be dependent on the observational setup and the strength of the model's nonlinear dynamics. It is also shown that, even in cases where the statistical error in some waves remains comparable with the stochastic EnKF during the DA stage, their prediction is improved remarkably when using the initial state resulting from the QPEns.

Model-based analysis and forecast of sleep-wake regulatory dynamics: Tools and applications to data.

Extensive clinical and experimental evidence links sleep-wake regulation and state of vigilance (SOV) to neurological disorders including schizophrenia and epilepsy. To understand the bidirectional coupling between disease severity and sleep disturbances, we need to investigate the underlying neurophysiological interactions of the sleep-wake regulatory system (SWRS) in normal and pathological brains. We utilized unscented Kalman filter based data assimilation (DA) and physiologically based mathematical models of a sleep-wake regulatory network synchronized with experimental measurements to reconstruct and predict the state of SWRS in chronically implanted animals. Critical to applying this technique to real biological systems is the need to estimate the underlying model parameters. We have developed an estimation method capable of simultaneously fitting and tracking multiple model parameters to optimize the reconstructed system state. We add to this fixed-lag smoothing to improve reconstruction of random input to the system and those that have a delayed effect on the observed dynamics. To demonstrate application of our DA framework, we have experimentally recorded brain activity from freely behaving rodents and classified discrete SOV continuously for many-day long recordings. These discretized observations were then used as the "noisy observables" in the implemented framework to estimate time-dependent model parameters and then to forecast future state and state transitions from out-of-sample recordings.

An Enhanced Motorway Control System for Mixed Manual/Automated Traffic Flow

Recent advances in vehicle automation technologies have opened up new perspectives for motorway traffic condition monitoring and macroscopic controls. When sufficient automated vehicles (AVs) are present, they can provide wide-ranging and spatiotemporally detailed traffic information. Furthermore, AVs can be programmed to comply with traffic laws and optimized to smooth traffic flows. Thus, deploying AVs are expected to address the issues encountered by the existing variable speed limit (VSL) systems. However, the appealing scenario that 90% or more AVs are on motorways is seen as a long-term goal. Therefore, this article explores the utilization of AV technologies in VSL systems under mixed traffic conditions where AVs coexist with manually driven vehicles. In this article, a VSL system using AVs as a source of data is presented. More specifically, an extended Kalman filter-based data assimilation method is proposed to estimate system variables (i.e., density, speed, critical density, and compliance rate) from the collected AV data. Following this, a model predictive control scheme is adopted to solve the optimal AV-data-based VSL control problem on the basis of estimated system variables. Finally, the efficiency of the proposed system is verified against a real motorway section in New Zealand.

Consistent Robust and Recursive Estimation of Atmospheric Motion Vectors From Satellite Images

Atmospheric motion vectors (AMVs) estimation helps in better understanding of atmospheric dynamics and also plays a key role in weather forecasting. It has been a challenging task because of the nonrigid motion of clouds and cyclones. In this paper, a modified Weighted Ensemble Transform Kalman Filter-based data assimilation technique is proposed for accurate flow vector estimation at each pixel directly from satellite generated infrared images of clouds/cyclones. This method provides clear visualization of both local and global motion with spatial and temporal consistencies very efficiently even in the case of splitting and merging of clouds or over long tracks. One of the key abilities of proposed method is in forecasting applications and also for generating motion vectors in the absence of data in real scenarios, even without the usage of the existing complex weather models. Estimated AMVs are validated using state-of-the-art European Centre for Medium-range Weather Forecasting (ECMWF) analysis data, and cyclone tracks are validated using the Indian Meteorological Department (IMD) best track data. The results obtained demonstrate the efficacy of proposed method over other existing methods.

Ensemble Kalman filter-based data assimilation for three-dimensional multi-phase-field model: Estimation of anisotropic grain boundary properties

Data assimilation (DA) has been used as a machine learning approach to estimate a system's state and the unknown parameters in its numerical model by integrating observed data into model predictions. In this paper, we propose using the DA methodology based on the ensemble Kalman filter (EnKF) to improve the accuracy of microstructure prediction using three-dimensional multi-phase-field (3D-MPF) model and estimate the model parameters simultaneously. To demonstrate the applicability of the DA methodology, we performed numerical experiments in which a priori assumed true parameters related to the grain boundary (GB) energy cusp and GB mobility peak of Σ7 coincidence site lattice GB were estimated from synthetic data of time-evolving polycrystalline microstructure. Four model parameters related to the Σ7 GB properties were successfully estimated by assimilating the synthetic microstructure data to the 3D-MPF model predictions using the EnKF-based DA method. Furthermore, we accurately reproduced the preliminarily assumed true shapes of GB energy cusp and GB mobility peak by using the estimated parameters. The results suggest that implementation of the EnKF-based DA method in the MPF model has great potential for identifying unknown material properties and estimating unmeasurable microstructure evolutions in polycrystalline materials based on real time-series 3D microstructure observation data.

Parallel simulation based adaptive prediction for equipment remaining useful life

The latest demands for remaining useful life (RUL) prediction are online prediction, real-time prediction and adaptive prediction. This paper addresses the demands of RUL prediction and proposes a novel framework of parallel simulation based adaptive prediction for equipment RUL. In the framework, a Wiener state space model (WSSM) is developed to achieve the aim, which considers the whole historical data and monitoring noise. Driven by the online observation data, the degradation state is estimated by the Kalman filter based data assimilation and the WSSM parameters are updated by the expectation maximum algorithm. An analytical RUL distribution considering the distribution of the degradation state is obtained based on the concept of the first hitting time. A case study for GaAs laser device is provided to substantiate the superiority of the proposed method compared with the competing method of traditional Wiener process. The results show that the parallel simulation method can provide better RUL prognostic accuracy.

Dynamic Evaluation on the Groundwater Flow Process in a River Basin Using the Ensemble Kalman Filter Method with Localization

Ordos Plateau is located in a semi-arid area in the northwest part of China, and groundwater water is the major water source to sustain the ecosystem. With the industrial construction and economic development, it is required to manage the groundwater usage reasonable. Several monitoring instruments have been installed to monitor the groundwater dynamics continuously. An ensemble Kalman filter based data assimilation method is developed to use the sequentially available monitoring data to characterize the groundwater system. To improve the algorithm stability and avoid the spurious correlation estimation, a localization function is introduced. The analysis results show the proposed EnKF method can reduce the estimation uncertainty and improve the predictability.

Integrating hydrological modelling, data assimilation and cloud computing for real-time management of water resources

Online data acquisition, data assimilation and integrated hydrological modelling have become more and more important in hydrological science. In this study, we explore cloud computing for integrating field data acquisition and stochastic, physically-based hydrological modelling in a data assimilation and optimisation framework as a service to water resources management. For this purpose, we developed an ensemble Kalman filter-based data assimilation system for the fully-coupled, physically-based hydrological model HydroGeoSphere, which is able to run in a cloud computing environment. A synthetic data assimilation experiment based on the widely used tilted V-catchment problem showed that the computational overhead for the application of the data assimilation platform in a cloud computing environment is minimal, which makes it well-suited for practical water management problems. Advantages of the cloud-based implementation comprise the independence from computational infrastructure and the straightforward integration of cloud-based observation databases with the modelling and data assimilation platform.

Ensemble Kalman filter based data assimilation in the Delft3D-BLOOM lake eutrophication model

Ensemble Kalman filter based data assimilation in the Delft3D-BLOOM lake eutrophication model

From skin to bulk: An adjustment technique for assimilation of satellite-derived temperature observations in numerical models of small inland water bodies

Data Assimilation (DA) has been proposed for multiple water resources studies that require rapid employment of incoming observations to update and improve accuracy of operational prediction models. The usefulness of DA approaches in assimilating water temperature observations from different types of monitoring technologies (e.g., remote sensing and in-situ sensors) into numerical models of in-land water bodies (e.g., lakes and reservoirs) has, however, received limited attention. In contrast to in-situ temperature sensors, remote sensing technologies (e.g., satellites) provide the benefit of collecting measurements with better X-Y spatial coverage. However, assimilating water temperature measurements from satellites can introduce biases in the updated numerical model of water bodies because the physical region represented by these measurements do not directly correspond with the numerical model's representation of the water column. This study proposes a novel approach to address this representation challenge by coupling a skin temperature adjustment technique based on available air and in-situ water temperature observations, with an ensemble Kalman filter based data assimilation technique. Additionally, the proposed approach used in this study for four-dimensional analysis of a reservoir provides reasonably accurate surface layer and water column temperature forecasts, in spite of the use of a fairly small ensemble. Application of the methodology on a test site - Eagle Creek Reservoir - in Central Indiana demonstrated that assimilation of remotely sensed skin temperature data using the proposed approach improved the overall root mean square difference between modeled surface layer temperatures and the adjusted remotely sensed skin temperature observations from 5.6°C to 0.51°C (i.e., 91% improvement). In addition, the overall error in the water column temperature predictions when compared with in-situ observations also decreased from 1.95°C (before assimilation) to 1.42°C (after assimilation), thereby, giving a 27% improvement in errors. In contrast, doing data assimilation without the proposed temperature adjustment would have increased this error to 1.98°C (i.e., 1.5% deterioration).

Assimilation of Active and Passive Microwave Observations for Improved Estimates of Soil Moisture and Crop Growth

An ensemble Kalman Filter-based data assimilation framework that links a crop growth model with active and passive (AP) microwave models was developed to improve estimates of soil moisture (SM) and vegetation biomass over a growing season of soybean. Complementarities in AP observations were incorporated to the framework, where the active observations were used to optimize surface roughness and update vegetation biomass, while passive observations were used to update SM. The framework was implemented in a rain-fed agricultural region of the southern La-Plata Basin during the 2011–2012 growing season, through a synthetic experiment and AP observations from the Aquarius mission. The synthetic experiment was conducted at a temporal resolution of 3 and 7 days to match the current AP missions. The assimilated estimates of SM in the root zone and dry biomass were improved compared to those from the cases without assimilation, during both 3- and 7-day assimilation scenarios. Particularly, the 3-day assimilation provided the best estimates of SM in the near surface and dry biomass with reductions in RMSEs of 41% and 42%, respectively. The absolute differences of assimilated LAI from Aquarius were $ compared to the MODIS LAI indicating that the performance of assimilation was similar to the MODIS product at a regional scale. This study demonstrates the potential of assimilation using AP observations at high temporal resolution such as those from soil moisture active passive (SMAP) for improved estimates of SM and vegetation parameters.

Evaluating atmospheric methane inversion model results for Pallas, northern Finland

A state-of-the-art inverse model, CarbonTracker Data Assimilation Shell (CTDAS), was used to optimize estimates of methane (CH4) surface fluxes using atmospheric observations of CH4 as a constraint. The model consists of the latest version of the TM5 atmospheric chemistry-transport model and an ensemble Kalman filter based data assimilation system. The model was constrained by atmospheric methane surface concentrations, obtained from the World Data Centre for Greenhouse Gases (WDCGG). Prior methane emissions were specified for five sources: biosphere, anthropogenic, fire, termites and ocean, of which bio-sphere and anthropogenic emissions were optimized. Atmospheric CH 4 mole fractions for 2007 from northern Finland calculated from prior and optimized emissions were compared with observations. It was found that the root mean squared errors of the posterior esti - mates were more than halved. Furthermore, inclusion of NOAA observations of CH 4 from weekly discrete air samples collected at Pallas improved agreement between posterior CH 4 mole fraction estimates and continuous observations, and resulted in reducing optimized biosphere emissions and their uncertainties in northern Finland.

A problem-solving environment for data assimilation in air quality modelling

A generic toolbox for data assimilation called COSTA (COmmon Set of Tools for the Assimilation of data) makes it possible to simplify the application of data assimilation to models and to try out various methods for a particular model. Concepts of object oriented programming are used to define building blocks for data assimilation systems that can be exchanged and reused. The main building blocks in COSTA are the model component and the stochastic observer component. These components can be created by wrapping existing code. The LOTOS–EUROS air quality model will be used for operational smog and aerosol forecasts in the Netherlands in the near future. The COSTA framework will be used in this operational environment to implement the data assimilation techniques. As a first steps towards this operational system the model component of LOTOS–EUROS is created and the performance of various Kalman filter based data assimilation techniques are compared for a real live case study. The ensemble Kalman filter and the ensemble square-root filters converged faster than the other tested techniques for the selected case study setup.

Dynamic Calibration with an Ensemble Kalman Filter Based Data Assimilation Approach for Root-Zone Moisture Predictions

AbstractIn the presence of uncertain initial conditions and soil hydraulic properties, land surface model (LSM) performance can be significantly improved by the assimilation of periodic observations of certain state variables, such as the near-surface soil moisture (θg), as observed from a remote platform. In this paper the possibility of merging observations and the model optimally for providing robust predictions of root-zone soil moisture (θ2) is demonstrated. An assimilation approach that assimilates θg through the ensemble Kalman filter (EnKF) and provides a physics-based update of θ2 is developed. This approach, as with other common soil moisture assimilation approaches, may fail when a key LSM parameter, for example, the saturated hydraulic conductivity (ks), is estimated poorly. This leads to biased model errors producing a violation of a main assumption (model errors with zero mean) of the EnKF. For overcoming this model bias an innovative assimilation approach is developed that accepts this violation in the early model run times and dynamically calibrates all the components of the ks ensemble as a function of the persistent bias in root-zone soil moisture, allowing one to remove the model bias, restore the fidelity to the EnKF requirements, and reduce the model uncertainty. The robustness of the proposed approach is also examined in sensitivity analyses.