Hybrid Data Assimilation Research Articles

A realtime observatory for laboratory simulation of planetary flows

Motivated by the mid-latitude atmospheric circulation, we develop a system that uses observations from a differentially heated rotating annulus experiment to constrain a numerical simulation in real-time. The coupled physical-numerical system provides a tool to rapidly prototype new methods for state and parameter estimation, and facilitates the study of prediction, predictability, and transport of geophysical fluids where observations or numerical simulations would not independently suffice. A computer vision system is used to extract measurements from the physical simulation, which constrain the model-state of the MIT general circulation model in a hybrid data assimilation approach. Using a combination of parallelism, domain decomposition and an efficient scheme to select ensembles of model-states, we show that estimates that effectively track the fluid-state can be produced. To the best of our knowledge, this is the first realtime coupled system for this laboratory analog of planetary circulation.

Efficient data assimilation method based on chaos theory and Kalman filter with an application in Singapore Regional Model

Data assimilation is a methodology that utilizes information from observations, and assimilates it into numerical models, with the intention of improving the quality and accuracy of the model outputs. This paper introduces a hybrid data assimilation scheme, which combines a temporal error prediction algorithm based on local model (LM) inspired by chaos theory and a spatial error distribution scheme through the propagation of error covariances derived from the Kalman filter (KF). Local model is only capable of predicting the model errors at the stations where the observations are available, while the effect of traditional Kalman filter is limited to a time horizon where the improved initial conditions are washed out. The hybrid scheme outlined in this paper is performed in two steps: (i) predicting the model errors at the measurement stations using the local model approach and (ii) distributing the predicted errors over the computational domain using the Kalman filter. Incorporating error distribution with error prediction assimilates limited information from the observed data into non-measurement stations. Therefore all stations of interest are able to be benefited. The proposed hybrid scheme has been implemented in the Singapore Regional Model (SRM) constructed by Delft3D modelling system, with the improvements in the assimilated characteristics discussed in detail.

A Hybrid ETKF–3DVAR Data Assimilation Scheme for the WRF Model. Part I: Observing System Simulation Experiment

Abstract A hybrid ensemble transform Kalman filter–three-dimensional variational data assimilation (ETKF–3DVAR) system for the Weather Research and Forecasting (WRF) Model is introduced. The system is based on the existing WRF 3DVAR. Unlike WRF 3DVAR, which utilizes a simple, static covariance model to estimate the forecast-error statistics, the hybrid system combines ensemble covariances with the static covariances to estimate the complex, flow-dependent forecast-error statistics. Ensemble covariances are incorporated by using the extended control variable method during the variational minimization. The ensemble perturbations are maintained by the computationally efficient ETKF. As an initial attempt to test and understand the newly developed system, both an observing system simulation experiment under the perfect model assumption (Part I) and the real observation experiment (Part II) were conducted. In these pilot studies, the WRF was run over the North America domain at a coarse grid spacing (200 km) to emphasize synoptic scales, owing to limited computational resources and the large number of experiments conducted. In Part I, simulated radiosonde wind and temperature observations were assimilated. The results demonstrated that the hybrid data assimilation method provided more accurate analyses than the 3DVAR. The horizontal distributions of the errors demonstrated the hybrid analyses had larger improvements over data-sparse regions than over data-dense regions. It was also found that the ETKF ensemble spread in general agreed with the root-mean-square background forecast error for both the first- and second-order measures. Given the coarse resolution, relatively sparse observation network, and perfect model assumption adopted in this part of the study, caution is warranted when extrapolating the results to operational applications.

The TOPAZ monitoring and prediction system for the Atlantic and Arctic Oceans

Ocean data assimilation systems allow the combination of remote sensing and in-situ ocean observations with primitive equation ocean general circulation models (OGCM). They thus provide initial conditions for short-term ocean currents forecasts (ten days to one month) and boundary conditions for nested high-resolution models of coastal seas and can be operated in hindcast to reproduce past events over long periods. At a time when offshore activities are moving toward deeper waters and ice-covered seas, accurate monitoring and forecasting of the environment (particularly ocean currents and seaice) cannot be neglected. This paper presents the TOPAZ system, being the Arctic component of the MERSEA integrated system and one of the contributors to the GODAE international initiative. The system is based on the latest scientific developments in terms of ocean modelling with the Hybrid Coordinate Ocean Model (HYCOM) and data assimilation with the Ensemble Kalman Filter (EnKF). Validation results of the system and applications in nested regional models are presented.

Soil Moisture Data Assimilation Using Support Vector Machines and Ensemble Kalman Filter1

Abstract: A hybrid data assimilation (DA) methodology that combines two state‐of‐the‐art techniques, support vector machines (SVMs) and ensemble Kalman filter (EnKF), is applied for soil moisture DA in this work. The SVM methodology provides a statistically sound and robust approach to solving the inverse problem, and thus to building statistical models. EnKF is an extension of the Kalman Filter (KF), a well‐known tool in prediction updating. In the present research, ground measurements were used to build a SVM‐type soil moisture predictor. Subsequent observations and their statistics were assimilated to update predictions from the SVM model by coupling it with EnKF. In this way, both model predictions and ground data, as well as their statistics, are fused thus minimizing the prediction error and making the predictions and observations statistically consistent. The results are shown for two approaches; one in which update is done at every time step and the other which assumes that data is only available at alternate time steps (in window of 10 time steps) and hence update is performed at those occasions. The SVM‐EnKF coupling is shown to improve soil moisture forecasts in an example using data from the Soil Climate Analysis Network site at Ames, Iowa.

Validation of a hybrid optimal interpolation and Kalman filter scheme for sea surface temperature assimilation

A hybrid data assimilation scheme designed for operational assimilation of satellite sea surface temperatures (SST) into an ocean model has been developed and validated against in-situ observations. The scheme consists of an optimal interpolation (OI) part and a greatly simplified Kalman filter (KF) part. The OI is performed only in the longitudinal and latitudinal directions. A climatological field is used as a background field for the interpolation. It is constructed by fitting daily averages of satellite SST to the annual mean, annual, and semiannual harmonics in a 20 km by 20 km grid. The background error covariance is approximated by a spatially varying two-dimensional exponential covariance model. The parameters of the covariance model are fitted to the deviations of the satellite data from the background field using data from a full year. The simplified KF uses ocean model forecasts as a background field. It is based on the assumption that it is possible to neglect horizontal SST covariances in the filter and that the typical time scale for vertical mixing in the mixed layer is much shorter than the average time between observations. We therefore assume that the error variance in a column of water is evenly spread out throughout the mixed layer. The result of these simplifications is a computationally very efficient KF. A one year validation of the scheme is performed for year 2001 using an operational eddy resolving ocean model covering the North Sea and the Baltic Sea. It is found that assimilation of sea surface temperature data reduces the model root mean square error from 1.13 °C to 0.70 °C. The hybrid scheme is found to reduce the root mean square error slightly more than the simplified KF without OI to 0.66 °C. The inclusion of spatially varying satellite error variances does not improve the performance of the scheme significantly.

Verification region selection and data assimilation for adaptive sampling

AbstractAdaptive or targeted observations supplement routine observations at a pre‐specified targeting time. Adaptive observation locations are selected to supplement routine observations in an attempt to minimize the forecast error variance of a future target forecast within some predefined verification region (VR) at some predefined verification time. Ideally, the VR is placed in a location where unusually large forecast errors are likely. Here, we compare three methods of selecting VRs: a climatological method based on seasonal averages of forecast errors; an unconditioned method based on verification time ensemble spread; and a conditioned method based on an ensemble transform Kalman filter (ETKF) estimate of forecast error variance given the routine observations to be taken at the targeting time. To test the effectiveness of the three approaches, observation‐system simulation experiments on a chaotic barotropic flow were performed using an imperfect model. To test the sensitivity of our results to the type of forecast error covariance model used in the data assimilation (DA) scheme, two types of DA schemes were tested: an isotropic and a hybrid scheme. For isotropic DA, correlations between vorticity forecast errors at any two points were solely a function of the distance between the points; for hybrid DA, the forecast error covariance matrix was a linear combination of the covariance matrix used in isotropic DA and the sample covariance matrix of an ETKF ensemble. For each of the three VRs, the ETKF was used to select two adaptive observations. To assess targeted‐observation‐induced error reductions, forecast errors with and without targeted observations were computed for the VR, the total domain and also for an empirical VR which was defined to be the VR with the largest forecast error without targeted observations. The sensitivity, to the type of DA scheme employed, of the ETKF's ability to distinguish high‐impact targeted observations from low‐impact observations, was also examined. Amongst other things, it was found that: conditioned VRs were more prone to large errors than the other VRs; and the targeted‐observation‐induced reduction in forecast error variance was largest when hybrid DA and conditioned VRs were used, as was the range of forecast impacts distinguished by ETKF predictions. Copyright © 2006 Royal Meteorological Society.

Adaptive Sampling with the Ensemble Transform Kalman Filter. Part I: Theoretical Aspects

A suboptimal Kalman filter called the ensemble transform Kalman filter (ET KF) is introduced. Like other Kalman filters, it provides a framework for assimilating observations and also for estimating the effect of observations on forecast error covariance. It differs from other ensemble Kalman filters in that it uses ensemble transformation and a normalization to rapidly obtain the prediction error covariance matrix associated with a particular deployment of observational resources. This rapidity enables it to quickly assess the ability of a large number of future feasible sequences of observational networks to reduce forecast error variance. The ET KF was used by the National Centers for Environmental Prediction in the Winter Storm Reconnaissance missions of 1999 and 2000 to determine where aircraft should deploy dropwindsondes in order to improve 24‐72-h forecasts over the continental United States. The ET KF may be applied to any well-constructed set of ensemble perturbations. The ET KF technique supercedes the ensemble transform (ET) targeting technique of Bishop and Toth. In the ET targeting formulation, the means by which observations reduced forecast error variance was not expressed mathematically. The mathematical representation of this process provided by the ET KF enables such things as the evaluation of the reduction in forecast error variance associated with individual flight tracks and assessments of the value of targeted observations that are distributed over significant time intervals. It also enables a serial targeting methodology whereby one can identify optimal observing sites given the location and error statistics of other observations. This allows the network designer to nonredundantly position targeted observations. Serial targeting can also be used to greatly reduce the computations required to identify optimal target sites. For these theoretical and practical reasons, the ET KF technique is more useful than the ET technique. The methodology is illustrated with observation system simulation experiments involving a barotropic numerical model of tropical cyclonelike vortices. These include preliminary empirical tests of ET KF predictions using ET KF, 3DVAR, and hybrid data assimilation schemes—the results of which look promising. To concisely describe the future feasible sequences of observations considered in adaptive sampling problems, an extension to Ide et al.’s unified notation for data assimilation is suggested.