Robust M-estimation Research Articles

Robust State Estimation Model for Low Voltage Distribution Networks in the Presence of Multiple Gross Errors

One of the primary challenges in solving the State Estimation (SE) problem in low voltage networks is the presence of Gross Errors (GE). If the SE model fails to accurately estimate state variables in the presence of GE, the system operator may receive a distorted image of the power network, potentially leading to unforeseen disruptions, blackouts, and significant economic losses. The classical weighted least squares method, which is commonly used in such problems, exhibits low accuracy when simultaneous GE occurs. This paper provides a comprehensive analysis comparing robust M-estimators designed to handle GE, such as Hachtel and the largest normalized residual test, and presents a novel SE method called the Adaptive Maximum Correntropy Criterion (AMCC). The AMCC employs the maximum correntropy criterion for the loss function and an interior point methodology. Additionally, an adaptive scheme is employed to automatically adjust a high parameter associated with the shape of the related gamma function. We show that, in a real low voltage network, the AMCC exhibits superior accuracy with smaller root-mean-square errors compared to the other estimators studied.

Improved regression in ratio type estimators based on robust M-estimation.

In this article, a new robust ratio type estimator using the Uk's redescending M-estimator is proposed for the estimation of the finite population mean in the simple random sampling (SRS) when there are outliers in the dataset. The mean square error (MSE) equation of the proposed estimator is obtained using the first order of approximation and it has been compared with the traditional ratio-type estimators in the literature, robust regression estimators, and other existing redescending M-estimators. A real-life data and simulation study are used to justify the efficiency of the proposed estimators. It has been shown that the proposed estimator is more efficient than other estimators in the literature on both simulation and real data studies.

Msplit Estimation Approach to Modeling Vertical Terrain Displacement from TLS Data Disturbed by Outliers

Terrestrial laser scanning (TLS) is a modern measurement technique that provides a point cloud in a relatively short time. TLS data are usually processed using different methods in order to obtain the final result (infrastructure or terrain models). Msplit estimation is a modern method successfully applied for such a purpose. This paper addresses the possible application of the method in processing TLS data from two different epochs to model a vertical displacement of terrain resulting, for example, from landslides or mining damages. Msplit estimation can be performed in two variants (the squared or absolute method) and two scenarios (two point clouds or one combined point cloud). One should understand that point clouds usually contain outliers of different origins. Therefore, this paper considers the contamination of TLS data by positive or/and negative outliers. The results based on simulated data prove that absolute Msplit estimation provides better results and overperforms conventional estimation methods (least-squares or robust M-estimation). In practice, the processing of point clouds separately seems to be a better option. This paper proved that Msplit estimation is a compelling alternative to conventional methods, as it can be applied to process TLS data disturbed by outliers of different types.

Proposed Hybrid Method for Wavelet Shrinkage with Robust Multiple Linear Regression Model With Simulation Study

This study compares the proposed hybrid method (wavelet robust M-estimation) to the traditional method (wavelet ordinary least square) when there are de-noising or outlier problems for estimating multiple linear regression models using the statistical criterion root mean square error (RMSE). According to simulated and real data, the proposed hybrid method (wavelet robust M-estimation) is better than the classical method (Wavelet Ordinary Least Square) and more accurate. The root mean square error of the proposed hybrid method (wavelet robust M-estimation) is less than the Wavelet Ordinary Least Square. Therefore, it is recommended to use the hybrid proposed method to reduce the problem of outliers and de-noise data.

Robust time‐series analysis of the effects of environmental factors on the CoViD‐19 pandemic in the area of Milan (Italy) in the years 2020–21

The effects of environmental factors on the spread of the CoViD-19 pandemic have been widely debated in the scientific literature. The results are important for understanding the outbreak dynamics and for defining health measures of prevention and containment. Using multivariate autoregressive (AR) models and robust statistics of causality, this paper analyzes the effect of 19 time series (10 physical and 9 social) on 3 daily CoViD-19 series (infected, hospitalized, deaths) in the Milan area for about 16 months. Robust M-estimation shows the weak effect of climatic and pollution factors, while authority restrictions, people mobility, smart working and vaccination rate have a significant impact. In particular, the vaccination campaign is important for reducing hospitalizations and deaths.

Proposed Hybrid Method for Wavelet Shrinkage with Robust Multiple Linear Regression Model

This study compares the proposed hybrid method (wavelet robust M-estimation) to the traditional method (wavelet ordinary least square) when there are de-noising or outlier problems for estimating multiple linear regression models using the statistical criterion root mean square error (RMSE). According to simulated and real data, the proposed hybrid method (wavelet robust M-estimation) is better than the classical method (Wavelet Ordinary Least Square) and more accurate. The root mean square error of the proposed hybrid method (wavelet robust M-estimation) is less than the Wavelet Ordinary Least Square. Therefore, it is recommended to use the hybrid proposed method to reduce the problem of outliers and de-noise data.

Robust symmetric iterative closest point

Point cloud registration (PCR) is an important technique of 3D vision, which has been widely applied in many areas such as robotics and photogrammetry. The iterative closest point (ICP) is a de facto standard for PCR. However, it mainly suffers from two drawbacks: small convergence basin and the sensitivity to outliers and partial overlaps. In this paper, we propose a robust symmetric ICP (RSICP) to tackle these drawbacks. First, we present a new symmetric point-to-plane distance metric whose functional zero-set is a set of locally-second-order surfaces. It has a wider convergence basin and higher convergence speed than the point-to-point metric, point-to-plane metric, and even original symmetric metric. Second, we introduce an adaptive robust loss to construct our robust symmetric metric. This robust loss bridges the gap between the non-robust ℓ2 cost and robust M-estimates. In the optimization, we gradually improve the degrees of robustness via the decay of a robustness control parameter. This loss has a high “breakdown” point or low computational overhead compared with recent work (e.g., Sparse ICP and Robust ICP). We also present a simple but effective linearization for the alignment function based on Rodrigues rotation parameterization with the small incremental rotation assumption. Extensive experiments on challenging datasets with noise, outliers or partial overlaps show that the proposed algorithm significantly outperforms Sparse ICP and Robust ICP in terms of both accuracy and efficiency. Our source code will be publicly available in https://ljy-rs.github.io/web.

Asymptotic normality of robust M-estimators with convex penalty

This paper develops asymptotic normality results for individual coordinates of robust M-estimators with convex penalty in high-dimensions, where the dimension p is at most of the same order as the sample size n, i.e, p∕n≤γ for some fixed constant γ>0. The asymptotic normality requires a bias correction and holds for most coordinates of the M-estimator for a large class of loss functions including the Huber loss and its smoothed versions regularized with a strongly convex penalty. The asymptotic variance that characterizes the width of the resulting confidence intervals is estimated with data-driven quantities. This estimate of the variance adapts automatically to low (p∕n→0) or high (p∕n≤γ) dimensions and does not involve the proximal operators seen in previous works on asymptotic normality of M-estimators. For the Huber loss, the estimated variance has a simple expression involving an effective degrees-of-freedom as well as an effective sample size. The case of the Huber loss with Elastic-Net penalty is studied in details and a simulation study confirms the theoretical findings. The asymptotic normality results follow from Stein formulae for high-dimensional random vectors on the sphere developed in the paper which are of independent interest.

Parametric Magnetotelluric Impedance Tensor Estimation

Conventional magnetotelluric (MT) impedance tensor estimation methods are generally based on robust M-estimator spectral analysis along with remote reference (RR) processing to reduce the estimation biases caused by cultural noise. Here, we propose a parametric spectral estimation and system identification-based impedance tensor estimation method and validate it with three representative data segments measured at several USArray sites. The proposed parametric method has the potential to give an accurate and robust estimate of the electromagnetic transfer impedance without an RR while maintaining a smooth and continuous response in accordance with prior physical knowledge. The approach considers the geophysical problem in addition to signal processing and system identification, providing a statistically efficient strategy for electromagnetic transfer impedance estimation of periods from 10 to 10⁴ s.

Resiliency Improvement of an AC/DC Power Grid with Embedded LCC-HVDC Using Robust Power System State Estimation

The growth of renewable energy generation in the power grid brings attention to high-voltage direct current (HVDC) transmission as a valuable solution for stabilizing the system. Robust hybrid power system state estimation could enhance the resilience of the control of these systems. This paper proposes a two-stage, highly robust least-trimmed squares (LTS)-based estimator. The first step combines the supervisory control and data acquisition (SCADA) measurements using the robust LTS-based estimator. The second step merges the obtained state results with the available phasor measurement units (PMUs) measurements using a robust Huber M-estimator. The proposed robust LTS-based estimator shows good performance in the presence of Gaussian measurement noise. The proposed estimator is shown to resist and correct the effect of false data injection (FDI) attacks and random errors on the measurement vector and the Jacobian matrix. The state estimation (SE) is executed on a modified version of the CIGRE bipole LCC-HVDC benchmark model integrated into the IEEE 12-bus AC dynamic test system. The obtained simulation results confirm the effectiveness and robustness of the proposed two-stage LTS-based SE.

Impact of cross-section centers estimation on the accuracy of the point cloud spatial expansion using robust M-estimation and Monte Carlo simulation

The point cloud spatial expansion (PCSE) method creates an alternative form of representing the shape of symmetrical objects and introduces additional descriptive geometric parameters. An important element of the procedure is determining the course of the axis of symmetry of cylindrical objects based on cross-sections of point clouds. Outliers occurring in laser measurements are of great importance in this case. In this study, six robust estimation methods were used to determine the coordinates of the section centers. Accuracy analysis was performed both for data simulated with the Monte Carlo method and the real data. The study showed the advantage of robust methods for the PCSE method over the classical method of least squares estimation.

High dimensional robust M-estimation : arbitrary corruption and heavy tails

We consider the problem of sparsity-constrained $M$-estimation when both explanatory and response variables have heavy tails (bounded 4-th moments), or a fraction of arbitrary corruptions. We focus on the $k$-sparse, high-dimensional regime where the number of variables $d$ and the sample size $n$ are related through $n \sim k \log d$. We define a natural condition we call the Robust Descent Condition (RDC), and show that if a gradient estimator satisfies the RDC, then Robust Hard Thresholding (IHT using this gradient estimator), is guaranteed to obtain good statistical rates. The contribution of this paper is in showing that this RDC is a flexible enough concept to recover known results, and obtain new robustness results. Specifically, new results include: (a) For $k$-sparse high-dimensional linear- and logistic-regression with heavy tail (bounded 4-th moment) explanatory and response variables, a linear-time-computable median-of-means gradient estimator satisfies the RDC, and hence Robust Hard Thresholding is minimax optimal; (b) When instead of heavy tails we have $O(1/\sqrt{k}\log(nd))$-fraction of arbitrary corruptions in explanatory and response variables, a near linear-time computable trimmed gradient estimator satisfies the RDC, and hence Robust Hard Thresholding is minimax optimal. We demonstrate the effectiveness of our approach in sparse linear, logistic regression, and sparse precision matrix estimation on synthetic and real-world US equities data.

Estimation of Feeding Composition of Industrial Process Based on Data Reconciliation.

For an industrial process, the estimation of feeding composition is important for analyzing production status and making control decisions. However, random errors or even gross ones inevitably contaminate the actual measurements. Feeding composition is conventionally obtained via discrete and low-rate artificial testing. To address these problems, a feeding composition estimation approach based on data reconciliation procedure is developed. To improve the variable accuracy, a novel robust M-estimator is first proposed. Then, an iterative robust hierarchical data reconciliation and estimation strategy is applied to estimate the feeding composition. The feasibility and effectiveness of the estimation approach are verified on a fluidized bed roaster. The proposed M-estimator showed better overall performance.

Performance comparison of variance models in a robust estimation method for heteroscedastic nonlinear models

Nonlinear regression models are commonly used in various fields such as toxicology/pharmacology. When analyzing data using a nonlinear regression model the structure of error variance plays a key role in the estimation of parameters. Particularly, when data do not satisfy the homoscedasticity assumption, it is important to use an appropriate estimation method. In this paper, a robust M-estimation method against potential outliers in nonlinear regression under heteroscedasticity is considered. Under the heteroscedasticity assumption, three variance models are considered, and a weighted M-estimator is studied by the simulation to compare the performance of the estimator with three variance models. From the results of the simulation studies, even though not as well as proper estimators, WME using a nonlinear variance model generally shows good performances for homoscedastic data and heteroscedastic data with the variance models. The methods are also illustrated by analyzing real toxicological data.

Robust Estimation for Poisson Integer-Valued GARCH Models Using a New Hybrid Loss

The Poisson integer-valued GARCH model is a popular tool in modeling time series of counts. The commonly used maximum likelihood estimator is strongly influenced by outliers, so there is a need to develop a robust M-estimator for this model. This paper has three aims. First, the authors propose a new loss function, which is a hybrid of the tri-weight loss for relatively small errors and the exponential squared loss for relatively large ones. Second, Mallows’ quasi-likelihood estimator (MQLE) is proposed as an M-estimator and its existence, uniqueness, consistency and asymptotic normality are established. In addition, a data-adaptive algorithm for computing MQLE is given based on a data-driven selection of tuning parameters in the loss function. Third, simulation studies and analysis of a real example are conducted to illustrate the performance of the new estimator, and a comparison with existing estimators is made.

Highly Efficient Robust and Stable M-Estimates of Location

This article is partially a review and partially a contribution. The classical two approaches to robustness, Huber’s minimax and Hampel’s based on influence functions, are reviewed with the accent on distribution classes of a non-neighborhood nature. Mainly, attention is paid to the minimax Huber’s M-estimates of location designed for the classes with bounded quantiles and Meshalkin-Shurygin’s stable M-estimates. The contribution is focused on the comparative performance evaluation study of these estimates, together with the classical robust M-estimates under the normal, double-exponential (Laplace), Cauchy, and contaminated normal (Tukey gross error) distributions. The obtained results are as follows: (i) under the normal, double-exponential, Cauchy, and heavily-contaminated normal distributions, the proposed robust minimax M-estimates outperform the classical Huber’s and Hampel’s M-estimates in asymptotic efficiency; (ii) in the case of heavy-tailed double-exponential and Cauchy distributions, the Meshalkin-Shurygin’s radical stable M-estimate also outperforms the classical robust M-estimates; (iii) for moderately contaminated normal, the classical robust estimates slightly outperform the proposed minimax M-estimates. Several directions of future works are enlisted.

Empirical analyses of robustness of the square Msplit estimation

Abstract The paper presents Msplit estimation as an alternative to methods in the class of robust M-estimation. The analysis conducted showed that Msplit estimation is highly efficient in the identification of observations encumbered by gross errors, especially those of small or moderate values. The classical methods of robust estimation provide then unsatisfactory results. Msplit estimation also shows high robustness to single gross errors of large values. The presented analysis of Msplit estimators’ robustness is of a chiefly empirical nature and is based on the example of a simulated levelling network and a real angular-linear network. Using the Monte Carlo method, mean success rates for outlier identification were determined and the courses of empirical influence functions were specified. The outcomes of the analysis were compared with the relevant values achieved via selected methods of robust M-estimation.

Privacy-Preserving Parametric Inference: A Case for Robust Statistics

Differential privacy is a cryptographically motivated approach to privacy that has become a very active field of research over the last decade in theoretical computer science and machine learning. In this paradigm, one assumes there is a trusted curator who holds the data of individuals in a database and the goal of privacy is to simultaneously protect individual data while allowing the release of global characteristics of the database. In this setting, we introduce a general framework for parametric inference with differential privacy guarantees. We first obtain differentially private estimators based on bounded influence M-estimators by leveraging their gross-error sensitivity in the calibration of a noise term added to them to ensure privacy. We then show how a similar construction can also be applied to construct differentially private test statistics analogous to the Wald, score, and likelihood ratio tests. We provide statistical guarantees for all our proposals via an asymptotic analysis. An interesting consequence of our results is to further clarify the connection between differential privacy and robust statistics. In particular, we demonstrate that differential privacy is a weaker stability requirement than infinitesimal robustness, and show that robust M-estimators can be easily randomized to guarantee both differential privacy and robustness toward the presence of contaminated data. We illustrate our results both on simulated and real data. Supplementary materials for this article are available online.

Iteratively reweighted total least squares for PEIV model

The weighted total least-squares (WTLS) estimate for the partial EIV (PEIV) model is very susceptible to outliers. In view of outliers in observations and coefficient matrix, a robust estimate of WTLS which makes using of iteratively reweighted technique (IRTLS) for the PEIV model is developed by combining a new two-step iterated algorithm of the WTLS estimate with the sensitivity-analysis based on the robust M-estimation. The uniformly most powerful test statistics are constructed to determine the down-weighting factor and the variance of unit weight is estimated by least median squares (LMS) method possessing high break-down point. Depended on the PEIV model, two different down-weighting schemes are presented. In the first scheme down-weighting is only implemented for the coefficient matrix and not for observations when the elements of the coefficient matrix are estimated, and the second scheme is contrary. The most attractive aspect is that the suggested computational formulae are the same with the traditional robust least-squares (LS) methods. The two-dimensional affine transformation and liner fitting example are analyzed, and some comparisons are performed with different available weight functions. The proposed approach (Scheme 1) proves to be a powerful tool in detecting outliers for the PEIV model.

Tukey's robust M-estimator for phase demodulation of interferograms with nonuniform shifts.

In this paper, we introduce an iterative scheme for phase demodulation of interferograms with nonuniformly spaced phase shifts. Our proposal consists of two stages: first, the phase map is obtained through a least squares fitting; second, the phase steps are retrieved using a statistical robust estimator. In particular, we use Tukey's biweighted M-estimator because it can cope with both noisy data and outliers in comparison with the ordinary least squares estimator. Furthermore, we provide the frequency description of the algorithm and the phase demodulation allowing us to analyze the procedure and estimation according to the frequency transfer function (FTF) formalism for phase-shifting algorithms. Results show that our method can accurately retrieve the phase map and phase shifts, and it converges by the 10th iteration.