Error Covariance Research Articles

Development and Case Study Evaluation of Filtered Ensemble-based Tangent Linear Model for the Operational EnVar Data Assimilation System

Abstract The ensemble-based tangent linear model (ETLM), which evolves the static background error covariance (BEC) in time in the data assimilation window, is beneficial for improving the analysis and avoids the need for developing the tangent linear forecast model and its adjoint as in four-dimensional variational data assimilation (4DVar). This study proposes to apply the filtered ETLM (FETLM), which is composed of ensemble perturbations at limited time slots and localized with a quasi-Gaussian filter in the way that avoids calculation of the inverse matrix. Tests with the Lorenz (1996) model showed that FETLM evolved the static BEC as in 4DVar and improved the analysis. In a case study with the FETLM implemented in the hourly-updated regional forecast system, the Rapid Refresh Forecast System, the imbalance of the analysis was mitigated due to the flow-dependent BEC. Moreover, the cross-variable covariance in the FETLM made it possible to assimilate radar reflectivity effectively even without hydrometeors as control variables.

Side Information Incorporated ϵ -Stealthy Attack Against Remote State Estimation.

This article concentrates on designing the $\epsilon $ -stealthy attack against remote state estimation in cyber-physical systems, where the intercepted information and the side information sensed by additional sensors are simultaneously exploited by the attacker. Combining the two types of information, a novel attack model based on corrupted innovation is proposed, and the attacked error covariance of the remote estimator is derived. Then, to maximize the error covariance, an analytical attack strategy is given. Compared with the existing results, the proposed attack exhibits significantly improved performance while requiring only one auxiliary filter. Finally, the numerical simulations verify the effectiveness of the results.

On the impact of observation‐error correlations in data assimilation, with application to along‐track altimeter data

Abstract Data assimilation involves estimating the state of a system by combining observations from various sources with a background estimate of the state. The weights given to the observations and background state depend on their specified error covariance matrices. Observation errors are often assumed to be uncorrelated, even though this assumption is inaccurate for many modern datasets such as those from satellite observing systems. As methods allowing for a more realistic representation of observation‐error correlations are emerging, our aim in this article is to provide insight on their expected impact in data assimilation. First, we use a simple idealised system to analyse the effect of observation‐error correlations on the spectral characteristics of the solution. Next, we assess the relevance of these results in a more realistic setting, in which simulated along‐track (nadir) altimeter observations with correlated errors are assimilated in a global ocean model using a three‐dimensional variational assimilation (3D‐Var) method. Correlated observation errors are modelled in the 3D‐Var system using a diffusion operator. When the correlation length‐scale of the observation error is small compared with that of the background error, inflating the observation‐error variances can mitigate most of the negative effects from neglecting the observation‐error correlations. Accounting for observation‐error correlations in this situation still outperforms variance inflation, since it allows small‐scale information in the observations to be extracted more effectively while accelerating the convergence of the minimisation. Conversely, when the correlation length‐scale of the observation error is large compared with that of the background error, the effect of observation‐error correlations cannot be approximated properly with variance inflation. However, the correlation model needs to be constructed carefully to ensure the minimisation problem is adequately conditioned so that a robust solution can be obtained. Practical ways to achieve this are discussed.

Privacy-Preserving Distributed Recursive Filtering for State-Saturated Systems with Quantization Effects

This paper addresses the problem of distributed recursive filtering for state-saturated systems in a networked communication environment. An output mask function is employed to safeguard the pri- vacy of interaction data during node exchange in sensor networks. Scaled uniform quantization is intro- duced to facilitate the digital communication and optimize the network resource usage. The primary objective of the study is to design a distributed recursive filter that ensures the filtering error covariance remains bounded over a finite horizon. Specifically, by using Riccati-like equations, an upper bound for the filtering error covariance is derived, which depends on the network topology, the output mask func- tion, and the quantization level. The desired gain matrix is then solved recursively. Finally, the effective- ness of the proposed filtering algorithm is demonstrated through a three-tank simulation example.

Multilevel Monte Carlo methods for ensemble variational data assimilation

Abstract. Ensemble variational data assimilation relies on ensembles of forecasts to estimate the background error covariance matrix B. The ensemble can be provided by an ensemble of data assimilations (EDA), which runs independent perturbed data assimilation and forecast steps. The accuracy of the ensemble estimator of B is strongly limited by the small ensemble size that is needed to keep the EDA computationally affordable. Here we investigate the potential of the multilevel Monte Carlo (MLMC) method, a type of multifidelity Monte Carlo method, to improve the accuracy of the standard Monte Carlo estimator of B while keeping the computational cost of ensemble generation comparable. MLMC exploits the availability of a range of discretization grids, thus shifting part of the computational work from the original assimilation grid to coarser ones. MLMC differs from the mere averaging of statistical estimators, as it ensures that no bias from the coarse-resolution grids is introduced in the estimation. The implications for ensemble variational data assimilation systems based on EDAs are discussed. Numerical experiments with a quasi-geostrophic model demonstrate the potential of the approach, as MLMC yields more accurate background error covariances and reduced analysis error. The challenges involved in cycling a multilevel variational data assimilation system are identified and discussed.

Robust Recursive Least-Squares Wiener Fixed-Interval Smoother Based on Innovation Approach in Linear Continuous-Time Stochastic Systems with Uncertainties

This study develops a robust recursive least-squares (RLS) Wiener fixed-interval (FI) smoother by exploiting covariance information for linear continuous-time systems that face uncertainties in both their system and observation matrices. Uncertainties in the state-space model cause degradations in the signal and observed values. The robust FI smoothing and filtering methods introduced do not assume that the system and the observation matrix have norm-bounded uncertainties. An observable companion form represents the state space model of the degraded signal. Robust RLS FI smoothing is to minimize the mean-square value of the smoothing errors of the system state over a fixed interval. Section 3 introduces an integral equation satisfied by the impulse response function that is optimal for robust FI smoothing estimation of the system state. An integral equation for the impulse response function, which provides a filtering estimate of the state of the degraded system, is also shown. Theorem 1 presents the robust RLS FI smoothing and filtering algorithm for the signal and the system state using covariance information. Theorem 2 presents the robust RLS Wiener (RLSW) FI smoothing and filtering algorithm for the signal and the system state. Robust RLS FI smoother outperforms robust RLS filter in estimation accuracy, as shown by the FI smoothing error covariance function in Section 5. Numerical simulation examples demonstrate that the robust RLSW FI smoother achieves superior signal estimation accuracy compared to the robust RLSW filter.

Pooled error variance and covariance estimation of sparse in situ soil moisture sensor measurements in agricultural fields in Flanders

Abstract. Accurately quantifying errors in soil moisture measurements from in situ sensors at fixed locations is essential for reliable state and parameter estimation in probabilistic soil hydrological modeling. This quantification becomes particularly challenging when the number of sensors per field or measurement zone (MZ) is limited. When direct calculation of errors from sensor data in a certain MZ is not feasible, we propose to pool systematic and random errors of soil moisture measurements for a specific measurement setup and derive a pooled error covariance matrix that applies to this setup across different fields and soil types. In this study, a pooled error covariance matrix was derived using soil moisture sensor measurements from three TEROS 10 (Meter Group, Inc., USA) sensors per MZ and soil moisture sampling campaigns conducted over three growing seasons, covering 93 cropping cycles in agricultural fields with diverse soil textures in Belgium. The MZ soil moisture estimated from a composite of nine soil samples with a small standard error (0.0038 m3 m−3) was considered the “true” MZ soil moisture. Based on these measurement data, we established a pooled linear recalibration of the TEROS 10 manufacturer's sensor calibration function. Then, for each individual sensor as well as for each MZ, we identified systematic offsets and temporally varying residual deviations between the calibrated sensor data and sampling data. Sensor deviations from the “true” MZ soil moisture were defined as observational errors and lump both measurement errors and representational errors. Since a systematic offset persists over time, it contributes to the temporal covariance of sensor observational errors. Therefore, we estimated the temporal covariance of observational errors of the individual and the MZ-averaged sensor measurements from the variance of the systematic offsets across all sensors and MZ averages, while the random error variance was derived from the variance of the pooled residual deviations. The total error variance was then obtained as the sum of these two components. Due to spatial soil moisture correlation, the variance and temporal covariance of MZ-averaged sensor observational errors could not be derived accurately from the individual sensor error variances and temporal covariances, assuming that the individual observational errors of the three sensors in a MZ were not correlated with each other. The pooled error covariance matrix of the MZ-averaged soil moisture measurements indicated a significant autocorrelation of sensor observational errors of 0.518, as the systematic error standard deviation (σα‾= 0.033 m3 m−3) was similar to the random error standard deviation (σϵ‾= 0.032 m3 m−3). To illustrate the impact of error covariance in probabilistic soil hydrological modeling, a case study was presented incorporating the pooled error covariance matrix in a Bayesian inverse modeling framework. These results demonstrate that the common assumption of uncorrelated random errors to determine parameter and model prediction uncertainty is not valid when measurements from sparse in situ soil moisture sensors are used to parameterize soil hydrological models. Further research is required to assess to what extent the error covariances found in this study can be transferred to other areas and how they impact parameter estimation in soil hydrological modeling.

A method for estimating and partitioning the covariance of wind speed observation error from GNSS reflectometry

Abstract This work introduces a mechanism for identifying and partitioning the sources of correlated error in wind speed observations made by spaceborne GNSS-reflectometry (GNSS-R). Observations by the CYGNSS constellation of GNSS-R satellites are matched with reanalysis wind speeds to determine the retrieval error. A correction for some of the error is constructed using ancillary environmental parameters. As a result of the corrections, the wind speed retrieval is improved by 11%. Further, we examine the correlated structure of both the wind speed error and the correction to provide physically plausible explanations for the correlated errors that are present in GNSS-R observations.

An Adaptive Variance Adjustment Strategy for a Static Background Error Covariance Matrix—Part I: Verification in the Lorenz-96 Model

Accurate initial conditions are crucial for improving numerical weather prediction (NWP). Variational data assimilation relies on a static background error covariance matrix (B), yet its variance estimation is often inaccurate, affecting assimilation and forecast performance. This study introduces DRL-AST, a deep reinforcement learning-based adaptive variance rescaling strategy that dynamically adjusts the variances of B to optimize forecast skill through improved assimilation performance. By formulating variance rescaling as a Markov Decision Process and employing an actor–critic framework with Proximal Policy Optimization, DRL-AST autonomously selects spatio-temporal rescaling factors, enhancing assimilation and forecast accuracy without additional computational cost. As a new paradigm for adaptive variance tuning, DRL-AST demonstrates competitive improvements in forecast skill in experiments with the Lorenz-96 model by generating initial states that better conform to model dynamical consistency. Given its adaptability and efficiency, DRL-AST holds great potential for application in high-dimensional NWP models, where deep learning-based dimensionality reduction and reinforcement learning techniques could further enhance its feasibility and effectiveness in complex assimilation frameworks.

Inaccuracy of the variance evolution associated with discrete covariance propagation

Abstract Data assimilation methods often also employ the same discrete dynamical model used to evolve the state estimate in time to propagate an approximation of the state estimation error covariance matrix. Four‐dimensional variational methods, for instance, evolve the covariance matrix implicitly via discrete tangent linear dynamics. Ensemble methods, while not forming this matrix explicitly, approximate its evolution at low rank from the evolution of the ensemble members. Such approximate evolution schemes for the covariance matrix imply an approximate evolution of the estimation error variances along its diagonal. For states that satisfy the advection equation, the continuity equation, or related hyperbolic partial differential equations (PDEs), the estimation error variance itself satisfies a known PDE, so the accuracy of the various approximations to the variances implied by the discrete covariance propagation can be determined directly. Experiments conducted by the atmospheric chemical constituent data assimilation community have indicated that such approximate variance evolution can be highly inaccurate. Through careful analysis and simple numerical experiments, we show that such poor accuracy must be expected, due to the inherent nature of discrete covariance propagation, coupled with a special property of the continuum covariance dynamics for states governed by these types of hyperbolic PDE. The intuitive explanation for this inaccuracy is that discrete covariance propagation involves approximating diagonal elements of the covariance matrix with combinations of off‐diagonal elements, even when there is a discontinuity in the continuum covariance dynamics along the diagonal. Our analysis uncovers the resulting error terms that depend on the ratio of the grid spacing to the correlation length, and these terms become very large when correlation lengths begin to approach the grid scale, for instance, as gradients steepen near the diagonal. We show that inaccurate variance evolution can manifest itself as both spurious loss and gain of variance.

Ionospheric Vertical Correlation Analysis and Modeling Using COSMIC‐2 Global Ionospheric Specification Data

Abstract The background error covariance matrix is essential in ionospheric data assimilation, enabling the integration of observational data with model predictions by defining spatial relationships between grid points. The vertical component plays a key role in enhancing model accuracy by capturing altitude‐dependent spatial correlations, directly influencing electron density profile reconstruction. In this study, we used Constellation Observing System for Meteorology, Ionosphere, and Climate‐2 Global Ionospheric Specification data to analyze vertical correlation distances in the ionosphere, identifying the altitude range where the correlation coefficient exceeds 0.75 in both upward and downward directions. The results revealed a clear pattern: vertical correlation distances initially increased with altitude, then the rate of increase slowed or decreased before rising again. Significant variations were observed with local time (LT), geomagnetic latitude, and seasonal effects. Winter anomalies, possibly due to changes in the O/N2 ratio, amplified vertical correlations, particularly between LT 12:00 and 18:00. A layered Gaussian model was developed to parameterize the vertical correlation coefficients, and its accuracy and adaptability were validated across different seasonal conditions. The model's correlation with observational data was greater than 0.75 across all seasons, with relatively small errors at lower altitudes, typically within 50 km. At reference heights exceeding 500 km, the error range remained within approximately 100 km for most points. These findings contribute to the development of more accurate ionospheric models, which are essential for satellite communication, navigation, and space weather forecasting.

Dynamic Event-Triggered Robust Fusion Estimation for Multi-Sensor Systems Under Time-Correlated Fading Channels

This paper investigates the problem of robust fusion state estimation for multi-sensor systems under the influence of time-correlated fading channels, incorporating a dynamic event-triggered mechanism (DETM). The randomly occurring parameter uncertainties are characterized by a stochastic variable following a Bernoulli distribution, while sensor measurements are transmitted to the corresponding estimators through time-correlated fading channels and dynamic event-triggered mechanisms. The DETM dynamically adjusts the triggering threshold via regulation and memory factors, enhancing adaptability in data transmission while effectively reducing redundant communication overhead. Furthermore, an augmented state model is constructed by integrating system states, channel coefficients, and the event-triggering mechanism, thereby comprehensively capturing the impact of dynamic environments on state estimation. Based on this model, a local state estimation algorithm is designed to ensure the convergence of the upper bound of the local estimation error covariance, which is further minimized at each time step through adaptive adjustment of local estimator gains. Subsequently, the local estimates obtained from multiple estimators are fused using the covariance intersection fusion strategy, improving the overall estimation accuracy. Simulation experiments demonstrate that the proposed recursive fusion state estimation framework significantly reduces communication overhead and enhances estimation performance in the presence of both time-correlated fading channels and randomly occurring parameter uncertainties, while maintaining an acceptable computational cost. Compared to the traditional Kalman filtering method, the proposed recursive fusion state estimation algorithm improves estimation accuracy by 58% while increasing computational time by only 32.4%. Additionally, the DETM effectively reduces communication frequency by 36.7%

A Novel Variational Bayesian Method Based on Student’s t Noise for Underwater Localization

In underwater environments, the presence of multipath effects can cause measurement outliers in acoustic sensors, leading to reduced estimation accuracy for integrated navigation. To address this issue, this paper proposes a sliding window variational Kalman filter based on Student’s t-distribution (SWVKF-ST) to improve state estimation accuracy. First, this method makes use of Student’s t-distribution to model heavy-tailed noise and adopts the inverse Wishart distribution as the prior for noise covariance, thereby enhancing robustness against heavy-tailed distributions. On this basis, the state variables and measurements within the sliding window are jointly estimated using the variational Bayesian framework, which helps mitigate the impact of unknown noise characteristics on state estimation. In addition, this method constructs multiple fading factors to prevent the degradation of estimation accuracy caused by excessive adjustment of the predicted error covariance matrix. Finally, the simulations and actual experiment validate that the SWVKF-ST outperforms the compared filters, achieving higher filtering precision and stronger robustness to outliers. The method effectively reduces the uncertainty in the measurement noise covariance matrix and demonstrates excellent adaptability in complex underwater environments.

Small Area Estimation under Square Root Transformed Fay-Herriot Model with Functional Measurement Error in Covariates

Small Area Estimation under Square Root Transformed Fay-Herriot Model with Functional Measurement Error in Covariates

Optimal collaborative multiple attack strategy under the energy constraint in cyber-physical systems

This study proposes the collaborative multiple attack strategy in cyber-physical systems (CPSs) under the energy constraint. Distinct from the relevant studies which only consider deception attacks or denial-of-service (DoS) attacks, the collaborative design of false data injection (FDI) attacks and DoS attacks is investigated, which is more general in the practical scenarios. The duration times for two types of attacks are limited due to the constraint of the attack energy. First, this work proposes an attack model which implements two types of attacks cooperatively. Then, under the proposed attack model, the attack performance is quantified by deriving the error covariance matrix, which is more intricate than the existing results since it involves more related terms that include the decision variables of the multiple attacks. Based on this, the attack design problem is converted into an optimisation problem with more constraints and decision variables. By analyzing the structure of the error covariance, it is proved that solving the optimisation problem is equivalent to step-wisely resolving the optimal distribution of FDI attacks and the optimal scheduling of multiple attacks without losing optimality. And then, the optimal distribution is obtained by utilising the Lagrange multiplier method, and the optimal scheduling is solved by 0-1 programming, such that the optimal attack strategy is obtained. Finally, the results are validated through the simulation examples.

Using Chemical Transport Model and Climatology Data as Backgrounds for Aerosol Optical Depth Spatial–Temporal Optimal Interpolation

A common approach to estimating the spatial–temporal distribution of atmospheric species properties is data assimilation. Data assimilation methods provide the best estimate of the required parameter by combining observations with appropriate prior information (background) that can include the model output, climatology data, or some other first guess. One of the relatively simple and computationally cheap data assimilation methods is optimal interpolation (OI). It estimates a value of interest through a weighted linear combination of observational data and background that is defined only once for the whole time interval of interest. Spatial–temporal OI (STOI) utilizes both spatial and temporal observational error covariance and background error covariance. This allows for filling in not only spatial, but also temporal gaps in observations. We applied STOI to daily mean aerosol optical depth (AOD) observations obtained at the European AERONET (Aerosol Robotic Network) sites with the use of the GEOS-Chem chemical transport model simulations and the AOD climatology data as backgrounds. We found that mean square errors in the estimate when using modeled data are comparable with those when using climatology data. Based on these results, we merged estimates obtained using modeled and climatology data according to their mean square errors. This allows for improving the AOD estimates in areas where observations are limited in space and time.

An Adaptive Robust Event-Triggered Variational Bayesian Filtering Method with Heavy-Tailed Noise.

Event-triggered state estimation has attracted significant attention due to the advantage of efficiently utilizing communication resources in wireless sensor networks. In this paper, an adaptive robust event-triggered variational Bayesian filtering method is designed for heavy-tailed noise with inaccurate nominal covariance matrices. The one-step state prediction probability density function and the measurement likelihood function are modeled as Student's t-distributions. By choosing inverse Wishart priors, the system state, the prediction error covariance, and the measurement noise covariance are jointly estimated based on the variational Bayesian inference and the fixed-point iteration. In the proposed filtering algorithm, the system states and the unknown covariances are adaptively updated by taking advantage of the event-triggered probabilistic information and the transmitted measurement data in the cases of non-transmission and transmission, respectively. The tracking simulations show that the proposed filtering method achieves good and robust estimation performance with low communication overhead.

Confirmatory Factor Analysis of Posttraumatic Growth Inventory – Short Version

ABSTRACT A short version of the Posttraumatic Growth Inventory (PTGI-SV) is described and examined in a sample of 436 U.S. parents who responded to an online cross-sectional survey about their well-being during the coronavirus disease (COVID-19) pandemic. The objective was to confirm a single-factor structure of the PTGI-SV as used to measure pandemic-related posttraumatic growth. Review of prior literature and theory suggested a unidimensional model and an asymptotically free-distribution estimation method for confirmatory factor analysis was most appropriate. A single-factor model with four error covariances acceptably fit the sample data. Limitations of the current study include sample homogeneity and operationalization differences between the PTGI-SV and the measure from which it was derived. Implications for future research, including item response theory and variance in the construct of compassion, are discussed.

Psychometric assessment of the breastfeeding self-efficacy scale-short form: An alternative german translation.

Psychometric assessment of the breastfeeding self-efficacy scale-short form: An alternative german translation.

Distributed secure estimation for interconnected systems against eavesdropping under energy harvesting constrained encryption.

Distributed secure estimation for interconnected systems against eavesdropping under energy harvesting constrained encryption.