Root Filter Research Articles

Data assimilation for combustion ignition delay time simulation using schlieren image velocimetry

This study sought to improve the accuracy of simulating spray penetration and combustion ignition delay by means of data assimilation (DA). The simulations were conducted using the Reynolds-averaged Navier–Stokes (RANS) equations and assimilating the schlieren image data. In DA, an ensemble square root filter (EnSRF) was used to build the statistical model, making the simulation results more accurate without any change in the governing equations. Recognizing that the spray-cone injection angle has a large effect on penetration, we created ensemble members with different injection angles. And we applied the two-component velocity distribution calculated via SIV and updated both velocity and temperature by using a DA statistical model derived from RANS ensemble simulations. The ignition delay time is generally known to vary even under the same experimental conditions because it is influenced by many factors. In this study, we attempted the transient DA-assisted RANS simulation to predict the ignition delay time even when the temporal resolution and accuracy of the observation data ware insufficient. Our trials offer an example of how a combination of techniques can be effectively used to assimilate experimental data obtained under restricted conditions.

Non-asymptotic analysis of ensemble Kalman updates: effective dimension and localization

Abstract Many modern algorithms for inverse problems and data assimilation rely on ensemble Kalman updates to blend prior predictions with observed data. Ensemble Kalman methods often perform well with a small ensemble size, which is essential in applications where generating each particle is costly. This paper develops a non-asymptotic analysis of ensemble Kalman updates, which rigorously explains why a small ensemble size suffices if the prior covariance has moderate effective dimension due to fast spectrum decay or approximate sparsity. We present our theory in a unified framework, comparing several implementations of ensemble Kalman updates that use perturbed observations, square root filtering and localization. As part of our analysis, we develop new dimension-free covariance estimation bounds for approximately sparse matrices that may be of independent interest.

A Regional multi-Air Pollutant Assimilation System (RAPAS v1.0) for emission estimates: system development and application

Abstract. Top-down atmospheric inversion infers surface–atmosphere fluxes from spatially distributed observations of atmospheric composition in order to quantify anthropogenic and natural emissions. In this study, we developed a Regional multi-Air Pollutant Assimilation System (RAPAS v1.0) based on the Weather Research and Forecasting–Community Multiscale Air Quality (WRF–CMAQ) modeling system model, the three-dimensional variational (3D-Var) algorithm, and the ensemble square root filter (EnSRF) algorithm. This system can simultaneously assimilate hourly in situ CO, SO2, NO2, PM2.5, and PM10 observations to infer gridded emissions of CO, SO2, NOx, primary PM2.5 (PPM2.5), and coarse PM10 (PMC) on a regional scale. In each data assimilation window, we use a “two-step” scheme, in which the emissions are inferred first and then input into the CMAQ model to simulate initial conditions (ICs) of the next window. The posterior emissions are then transferred to the next window as prior emissions, and the original emission inventory is only used in the first window. Additionally, a “super-observation” approach is implemented to decrease the computational costs, observation error correlations, and influence of representative errors. Using this system, we estimated the emissions of CO, SO2, NOx, PPM2.5, and PMC in December and July 2016 over China using nationwide surface observations. The results show that compared to the prior emissions (2016 Multi-resolution Emission Inventory for China – MEIC 2016)), the posterior emissions of CO, SO2, NOx, PPM2.5, and PMC in December 2016 increased by 129 %, 20 %, 5 %, 95 %, and 1045 %, respectively, and the emission uncertainties decreased by 44 %, 45 %, 34 %, 52 %, and 56 %, respectively. With the inverted emissions, the RMSE of simulated concentrations decreased by 40 %–56 %. Sensitivity tests were conducted with different prior emissions, prior uncertainties, and observation errors. The results showed that the two-step scheme employed in RAPAS is robust in estimating emissions using nationwide surface observations over China. This study offers a useful tool for accurately quantifying multi-species anthropogenic emissions at large scales and in near-real time.

Hybrid Assimilation of Snow Cover Improves Land Surface Simulations over Northern China

Abstract Ensemble data assimilation (DA) is an efficient approach to reduce snow simulation errors by combining observation and land surface modeling. However, there is a small spread between ensemble members of simulated snowpack, which typically occurs for a long time with 100% snow cover fraction (SCF) or snow-free conditions. Here, we apply a hybrid DA method, in which direct insertion (DI) is a supplement of the ensemble square root filter (EnSRF), to assimilate the spaceborne SCF into a land surface model, driven by China Meteorological Administration Land Data Assimilation System high-resolution climate forcings over northern China during the snow season in 2021/22. Compared to the open-loop experiment (without SCF assimilation), the root-mean-square error (RMSE) of SCF is reduced by 6% through the original EnSRF and is even lower (by 14%) in the combined DI and EnSRF (EnSRFDI) experiment. The results reveal the ability of both EnSRF and EnSRFDI to improve the SCF estimation over regions where the snow cover is low, while only EnSRFDI is able to efficiently reduce the RMSE over areas with high SCF. Moreover, the SCF assimilation is also observed to improve the snow depth and soil temperature simulations, with the Kling–Gupta efficiency (KGE) increasing at 60% and 56%–70% stations, respectively, particularly under conditions with near-freezing temperature, in which reliable simulations are typically challenging. Our results demonstrate that the EnSRFDI hybrid method can be applied for the assimilation of spaceborne observational snow cover to improve land surface simulations and snow-related operational products. Significance Statement Due to the small spread between the seasonal snowpack of ensemble simulations, ensemble snow cover fraction (SCF) data assimilation (DA) proves to be ineffective. Therefore, we apply a hybrid method that combines the direct insertion (DI) and ensemble square root filter (EnSRF) to assimilate the spaceborne SCF into a land surface model (LSM) driven by high-resolution climate forcings. Our results reveal the applicability of the EnSRFDI to further improve snow cover simulations over regions with high SCF. Furthermore, the DA experiments were validated through a large number of in situ observations from the China Meteorological Administration. The uncertainties of snow depth and soil temperature simulations are also slightly reduced by the SCF DAs, particularly over regions with a poor LSM performance.

An economical assimilation scheme for radar reflectivity in non‐convective region to suppress spurious precipitation

AbstractIncreasing convection information in initial field and weakening spurious convection information are hot topics in numerical weather prediction. How to economically assimilate radar reflectivity in the non‐convective region (NCR) is the focus of this study. This study proposes a new assimilation scheme for the two‐dimensional (2D) composite reflectivity in NCR through the ensemble square‐root filter method. A single‐column observation test and two convective cases are studied to verify the assimilation effect. Three experiments are designed for each test, including a control experiment (Exp_CTL) that only assimilates convective reflectivity and two assimilation experiments that assimilate reflectivity in NCR (one assimilates 3D weak reflectivity [Exp_RF] and the other assimilates 2D composite reflectivity [Exp_CRF]) based on the Exp_CTL. The results of real‐case studies show that the new scheme has two clear advantages. One is that it can save approximately three quarters of the assimilation workload. The other is that the new scheme has the most significant effect on weakening spurious convection and decreasing the false alarm ratios of precipitation and reflectivity.

Performance comparisons of the three data assimilation methods for improved predictability of PM2·5: Ensemble Kalman filter, ensemble square root filter, and three-dimensional variational methods

To improve the predictability of concentrations of atmospheric particulate matter, a data assimilation (DA) system using ensemble square root filter (EnSRF) has been developed for the Community Multiscale Air Quality (CMAQ) model. The EnSRF DA method is a deterministic variant of the ensemble Kalman filter (EnKF) method, which means that unlike the EnKF method, it does not add random noise to the observations. To compare the performances of the EnSRF with those of other DA methods, such as EnKF and 3DVAR (three-dimensional variational), these three methods were applied to the same CMAQ model simulations with identical experimental settings. This is the first attempt in the field of chemical DA to compare the EnKF and EnSRF methods. An identical set of surface fine particulate matter (PM2.5) were assimilated every 6 h by all the DA methods over a CMAQ domain of East Asia, during the period from 01 May to 11 June 2016. In parallel with ‘reanalysis experiments’, we also carried out ‘48 h prediction experiments’ using the optimized initial conditions produced by the three DA methods. Detailed analyses among the three DA methods were then carried out by comparing both the reanalysis and the prediction outputs with the observed surface PM2.5 over four regions (i.e., South Korea, the Beijing–Tianjin–Hebei (BTH) region, Shandong province, and Liaoning province). The comparison results revealed that the EnSRF produced the best reanalysis and prediction fields in terms of several statistical metrics. For example, when the 3DVAR, EnKF, and EnSRF methods were used, averaged normalized mean biases (NMBs) decreased by (57.6, 85.6, and 91.8) % in reanalyses and (39.7, 87.6, and 91.5) % in first-day predictions, compared to the CMAQ control experiment (i.e., without DA) over South Korea, respectively. Also, over the three Chinese regions, the EnSRF method outperformed the EnKF and 3DVAR methods.

Generalised homomorphic and root filtering in 2D-nonseparable discrete linear canonical transform domains in the image enhancement applications

Generalised homomorphic and root filtering in 2D-nonseparable discrete linear canonical transform domains in the image enhancement applications

Deep Learning Augmented Data Assimilation: Reconstructing Missing Information with Convolutional Autoencoders

Abstract Remote sensing data play a critical role in improving numerical weather prediction (NWP). However, the physical principles of radiation dictate that data voids frequently exist in physical space (e.g., subcloud area for satellite infrared radiance or no-precipitation region for radar reflectivity). Such data gaps impair the accuracy of initial conditions derived from data assimilation (DA), which has a negative impact on NWP. We use the barotropic vorticity equation to demonstrate the potential of deep learning augmented data assimilation (DDA), which involves reconstructing spatially complete pseudo-observation fields from incomplete observations and using them for DA. By training a convolutional autoencoder (CAE) with a long simulation at a coarse “forecast” resolution (T63), we obtained a deep learning approximation of the “reconstruction operator,” which maps spatially incomplete observations to a model state with full spatial coverage and resolution. The CAE was applied to an incomplete streamfunction observation (∼30% missing) from a high-resolution benchmark simulation and demonstrated satisfactory reconstruction performance, even when only very sparse (1/16 of T63 grid density) observations were used as input. When only spatially incomplete observations are used, the analysis fields obtained from ensemble square root filter (EnSRF) assimilation exhibit significant error. However, in DDA, when EnSRF takes in the combined data from the incomplete observations and CAE reconstruction, analysis error reduces significantly. Such gains are more pronounced with sparse observation and small ensemble size because the DDA analysis is much less sensitive to observation density and ensemble size than the conventional DA analysis, which is based solely on incomplete observations. Significance Statement Data assimilation plays a critical role in improving the skills of modern numerical weather prediction by establishing accurate initial conditions. However, unobservable regions are common in observation data, particularly those derived from remote sensing. The nonlinear relationship between data from observable regions and the physical state of unobservable regions may impede DA efficiency. As a result, we propose that deep learning be used to improve data assimilation in such cases by reconstructing a spatially complete first guess of the physical state with deep learning and then applying data assimilation to the reconstructed field. Such deep learning augmentation is found effective in improving the accuracy of data assimilation, especially for sparse observation and small ensemble size.

Observation system simulation experiments (OSSEs) for assimilation of the planetary boundary‐layer height (PBLH) using the EnSRF technique

AbstractThe planetary boundary‐layer height (PBLH) is a key parameter that is very important in numerical weather and air quality predictions. The current LiDAR networks make it possible to provide potential PBLH observations, and assimilating the parameter will be helpful to improve the forecast of variables within the planetary boundary layer (PBL). This study first carried out idealized experiments on PBLH assimilation through observation system simulation experiments (OSSEs). The ensemble square root filter (EnSRF) is applied to assimilate the simulated PBLH based on the Weather Research and Forecast (WRF) model. This study mainly focused on two issues: which variables can be effectively improved by assimilating the PBLH, and whether there are differences in the assimilation effects above and within the PBL in the vertical direction. The results show that during the daytime within the PBL, PBLH assimilation could effectively improve the simulation and forecasting of model variables, including perturbation potential temperature (pt), water vapour mixing ratio (qv) and perturbation geopotential (ph); meanwhile, PBLH assimilation could strengthen turbulence mixing and result in a warmer and drier PBL. While the vertical and horizontal wind speeds (w, u and v) cannot be effectively updated by assimilating the PBLH, they could be indirectly improved by model dynamical adjustment when pt, qv and ph are improved. Above the PBL top, most of the above six variables cannot be effectively improved by assimilating the daytime PBLH. In contrast, at night, only u and v within the PBL could be directly updated by the low nocturnal PBLH. In this study, the simplest and most idealized schemes are selected based on idealized experiments; nevertheless, the results lay a foundation for later vertical and horizontal localization research and for the study of LiDAR‐retrieved PBLH assimilation in the future, which is the ultimate goal of this series of works.

A Scheme to Suppress Spurious Convection by Assimilating the "Zero" Column Maximum Vertical Velocity

AbstractSuppressing spurious precipitation is one of the key issues in numerical weather prediction. To thoroughly suppress spurious convection, it is necessary to both inhibit the development of convection in false convection areas and weaken the heat and momentum exchange between false convection areas and other surrounding regions. In this study, a scheme to suppress spurious convection through cyclically assimilating the "zero" column maximum vertical velocity (wmax) using the ensemble square root filter (EnSRF) method is proposed. The "zero" wmax is the average wmax over the no‐rain echo region in the background field which is normally close to 0 m s−1. Two convective events that exhibit different characteristics are studied. The results show that the assimilation of "zero" wmax can suppress the upward motion and reduce the water vapor and hydrometeors of the background field in the false convection region, which can effectively suppress the development of spurious convection. In addition, the scheme can reduce the water vapor transport from surrounding areas to the spurious convection region and weaken the cold pool on the ground, which is not conducive to the regeneration and maintenance of false convection. The forecasts of reflectivity and precipitation show that the assimilation of "zero" wmax can effectively suppress either the strong false convection band or the weak spurious convection surrounding the true rain band and can decrease the false alarm rate of reflectivity and precipitation.

Length Scale Analyses of Background Error Covariances for EnKF and EnSRF Data Assimilation

Data assimilation (DA) combines incomplete background values obtained via chemical transport model predictions with observational information. Several 3-Dimensional variational (3DVAR) and sequential methods (e.g., ensemble Kalman filter (EnKF)) are used to define model errors and build a background error covariance (BEC) and are important factors affecting the prediction performance of DA. The BEC determines the spatial range, where observation concentration is reflected in the model when DA is applied to an air pollution transport model. However, studies investigating the characteristics of BEC using air quality models remain lacking. In this study, horizontal length scale (HLS) and vertical length scale (VLS) analyses of a BEC were applied to EnKF and ensemble square root filter (EnSRF), respectively, and two ensemble-based DA methods were performed; the characteristics were compared with those of a BEC applied to 3DVAR. The results of 6 h PM2.5 predictions performed for 42 days were evaluated for a control run without DA (CTR), 3DVAR, EnKF, and EnSRF. HLS and VLS respectively exhibited a high correlation with the ground wind speed and with the planetary boundary layer height for diurnal and daily variations; EnKF and EnSRF exhibited superior performances among all the methods. The root mean square errors were 11.9 μg m−3 and 11.7 μg m−3 for EnKF and EnSRF, respectively, while those for 3DVAR and CTR were 12.6 μg m−3 and 18.3 μg m−3, respectively. Thus, we proposed a simple method to find a Gaussian function that best described the error correlation of the BEC based on the physical distance.

Generalised homomorphic and root filtering in 2D-nonseparable discrete linear canonical transform domains in the image enhancement applications

Generalised homomorphic and root filtering in 2D-nonseparable discrete linear canonical transform domains in the image enhancement applications

Derivation of ensemble Kalman–Bucy filters with unbounded nonlinear coefficients

We provide a rigorous derivation of the ensemble Kalman–Bucy filter as well as the ensemble transform Kalman–Bucy filter in case of nonlinear, unbounded model and observation operators. We identify them as the continuous time limit of the discrete-time ensemble Kalman filter and the ensemble square root filters, respectively, together with concrete convergence rates in terms of the discretisation step size. Simultaneously, we establish well-posedness as well as accuracy of both the continuous-time and the discrete-time filtering algorithms.

Vehicle State Estimation Using Interacting Multiple Model Based on Square Root Cubature Kalman Filter

The distributed drive arrangement form has better potential for cooperative control of dynamics, but this drive arrangement form increases the parameter acquisition workload of the control system and increases the difficulty of vehicle control accordingly. In order to observe the vehicle motion state accurately and in real-time, while reducing the effect of uncertainty in noise statistical information, the vehicle state observer is designed based on interacting multiple model theory with square root cubature Kalman filter (IMM-SCKF). The IMM-SCKF algorithm sub-model considers different state noise and measurement noise, and the introduction of the square root filter reduces the complexity of the algorithm while ensuring accuracy and real-time performance. To estimate the vehicle longitudinal, lateral, and yaw motion states, the algorithm uses a three degree of freedom (3-DOF) vehicle dynamics model and a nonlinear brush tire model, which is then validated in a Carsim-Simulink co-simulation platform for multiple operating conditions. The results show that the IMM-SCKF algorithm’s fusion output results can effectively follow the sub-model with smaller output errors, and that the IMM-SCKF algorithm’s results are superior to the traditional SCKF algorithm’s results.

High Dynamic Weak Signal Tracking Algorithm of a Beidou Vector Receiver Based on an Adaptive Square Root Cubature Kalman Filter.

Compared with a scalar tracking receiver, the Beidou vector tracking receiver has the advantages of smaller tracking errors, fast loss-of-lock reacquisition, and high stability. However, in extremely challenging conditions, such as highly dynamic and weak signals, the loop will exhibit a high degree of nonlinearity, and observations with gross errors and large deviations will reduce the positioning accuracy and stability. In view of this situation, based on the concepts of cubature Kalman filtering and square root filtering, a square root cubature Kalman filtering (SRCKF) algorithm is given. Then, combining this algorithm with the concept of covariance matching based on an innovation sequence, an adaptive square root cubature Kalman filter (ASRCKF) algorithm is proposed. The algorithm was verified, and the tracking performance of the vector locking loop (VLL) realized by the algorithm was compared with the SRCKF VLL and the ASRCKF scalar locking loop (SLL). The simulation results show that, regardless of whether in a highly dynamic weak signal environment or in a general situation where the signal-to-noise ratio is higher than the tracking threshold, the tracking accuracy and stability of the ASRCKF VLL are higher than those of the SRCKF VLL and the ASRCKF SLL, the three-dimensional position error of the ASRCKF VLL does not exceed 36 m, and the three-dimensional velocity error does not exceed 3.5 m/s.

Machine learning techniques to construct patched analog ensembles for data assimilation

Using generative models from the machine learning literature to create artificial ensemble members for use within data assimilation schemes has been introduced in Grooms (2021) [1] as constructed analog ensemble optimal interpolation (cAnEnOI). Specifically, we study general and variational autoencoders for the machine learning component of this method, and combine the ideas of constructed analogs and ensemble optimal interpolation in the data assimilation piece. To extend the scalability of cAnEnOI for use in data assimilation on complex dynamical models, we propose using patching schemes to divide the global spatial domain into digestible chunks. Using patches makes training the generative models possible and has the added benefit of being able to exploit parallelism during the generative step. Testing this new algorithm on a 1D toy model, we find that larger patch sizes make it harder to train an accurate generative model (i.e. a model whose reconstruction error is small), while conversely the data assimilation performance improves at larger patch sizes. There is thus a sweet spot where the patch size is large enough to enable good data assimilation performance, but not so large that it becomes difficult to train an accurate generative model. In our tests the new patched cAnEnOI method outperforms the original (unpatched) cAnEnOI, as well as the ensemble square root filter results from [1].

Assimilation of the Maximum Vertical Velocity Converted From Total Lightning Data Through the EnSRF Method

AbstractA total lightning data assimilation (LDA) scheme is developed at the cloud‐resolving scale in this study. The LDA scheme assimilates total lightning data through the ensemble square root filter (EnSRF) method based on the relationship between the maximum vertical velocity and the instantaneous flash rate. To verify the effect of LDA, three assimilation experiments based on a convective activity on July 6, 2019 are performed. The results show that all LDA experiments can improve the forecast. LDA improves the water vapor field and provides a warm and moist environment in which convection can develop. LDA can also increase the convergence at low levels and the divergence at upper levels, which leads to intense convection development. However, the LDA experiment that does not consider vertical localization (LDA_no) shows excessive adjustments of the model variables, which leads to overestimation of precipitation. The LDA experiment which adjusts the state variables in the charge zone (LDA_CZ) underestimates the forecast. The LDA experiment using a global group filter (GGF) localization function (LDA_GGF) is optimal in theory and produces a better forecast. The results of LDA_GGF show that the interaction between the cold pool and the strong low‐level vertical wind shear in front of the cold pool is beneficial for triggering new convection in front of the convection, which leads to a rapid shifting in convective activity. In addition, the LDA_GGF scheme is evaluated further for a rainfall process on August 1, 2020 and obtains similar forecast improvements for composite reflectivity and accumulated precipitation.

Model-driven optimization of coastal sea observatories through data assimilation in a finite element hydrodynamic model (SHYFEM v. 7_5_65)

Abstract. Monitoring networks aims at capturing the spatial and temporal variability of one or several environmental variables in a specific environment. The optimal placement of sensors in an ocean or coastal observatory should maximize the amount of collected information and minimize the development and operational costs for the whole monitoring network. In this study, the problem of the design and optimization of ocean monitoring networks is tackled throughout the implementation of data assimilation techniques in the Shallow water HYdrodynamic Finite Element Model (SHYFEM). Two data assimilation methods – nudging and ensemble square root filter – have been applied and tested in the Lagoon of Venice (Italy), where an extensive water level monitoring network exists. A total of 29 tide gauge stations were available, and the assimilation of the observations results in an improvement of the performance of the SHYFEM model, which went from an initial root mean square error (RMSE) on the water level of 5.8 cm to a final value of about 2.1 and 3.2 cm for each of the two data assimilation methods. In the monitoring network optimization procedure, by excluding just one tide gauge at a time and always the station that contributes less to the improvement of the RMSE, a minimum number of tide gauges can be found that still allow for a successful description of the water level variability. Both data assimilation methods allow identifying the number of stations and their distribution that correctly represent the state variable in the investigated system. However, the more advanced ensemble square root filter has the benefit of keeping a physically and mass-conservative solution of the governing equations, which results in a better reproduction of the hydrodynamics over the whole system. In the case of the Lagoon of Venice, we found that, with the help of a process-based and observation-driven numerical model, two-thirds of the monitoring network can be dismissed. In this way, if some of the stations must be decommissioned due to a lack of funding, an a priori choice can be made, and the importance of a single monitoring site can be evaluated. The developed procedure may also be applied to the continuous monitoring of other ocean variables, like sea temperature and salinity.

Mean field limit of Ensemble Square Root filters - discrete and continuous time

<p style='text-indent:20px;'>Consider the class of Ensemble Square Root filtering algorithms for the numerical approximation of the posterior distribution of nonlinear Markovian signals, partially observed with linear observations corrupted with independent measurement noise. We analyze the asymptotic behavior of these algorithms in the large ensemble limit both in discrete and continuous time. We identify limiting mean-field processes on the level of the ensemble members, prove corresponding propagation of chaos results and derive associated convergence rates in terms of the ensemble size. In continuous time we also identify the stochastic partial differential equation driving the distribution of the mean-field process and perform a comparison with the Kushner-Stratonovich equation.</p>

On the continuous time limit of ensemble square root filters

We provide a continuous time limit analysis for the class of Ensemble Square Root Filter algorithms with deterministic model perturbations. In the particular linear case, we specify general conditions on the model perturbations implying convergence of the empirical mean and covariance matrix towards their respective counterparts of the Kalman-Bucy Filter. As a second main result we identify additional assumptions for the convergence of the whole ensemble towards solutions of the Ensemble Kalman-Bucy filtering equations introduced in [1]. The latter result can be generalized to nonlinear Lipschitz-continuous model operators. A striking implication of our results is the fact that the limiting equations for the ensemble members are universal for a large class of Ensemble Square Root Filters. This yields a mathematically rigorous justification for the analysis of these algorithms with the help of the Ensemble Kalman-Bucy Filter. [1] de Wiljes, Jana, Reich, Sebastian, Stannat, Wilhelm, Long-Time Stability and Accuracy of the Ensemble Kalman-Bucy Filter for Fully Observed Processes and Small Measurement Noise. SIAM Journal on Applied Dynamical Systems, Vol. 17, No. 2, 1152-1181, 2018