Bad Leverage Points Research Articles

Weighted fast-trimmed likelihood estimator for mixture regression models

The fast-trimmed likelihood estimate is a robust method to estimate the parameters of a mixture regression model. However, this method is vulnerable to the presence of bad leverage points, which are outliers in the direction of independent variables. To address this issue, we propose the weighted fast-trimmed likelihood estimate to mitigate the impact of leverage points. The proposed method applies the weights of the minimum covariance determinant to the rows suspected of containing leverage points. Notably, both real data and simulation studies were considered to determine the efficiency of the proposed method compared to the previous methods. The results reveal that the weighted fast-trimmed estimate method is more robust and reliable than the fast-trimmed likelihood estimate and the expectation–maximization (EM) methods, particularly in cases with small sample sizes.

A Robust Data-Driven Process Modeling Applied to Time-Series Stochastic Power Flow

In this paper, we propose a robust data-driven process model whose hyperparameters are robustly estimated using the Schweppe-type generalized maximum likelihood estimator. The proposed model is trained on recorded time-series data of voltage phasors and power injections to perform a time-series stochastic power flow calculation. Power system data are often corrupted with outliers caused by large errors, fault conditions, power outages, and extreme weather, to name a few. The proposed model downweights vertical outliers and bad leverage points in the measurements of the training dataset. The weights used to bound the influence of the outliers are calculated using projection statistics, which are a robust version of Mahalanobis distances of the time series data points. The proposed method is demonstrated on the IEEE 33-Bus power distribution system and a real-world unbalanced 240-bus power distribution system heavily integrated with renewable energy sources. Our simulation results show that the proposed robust model can handle up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$25\%$</tex-math></inline-formula> of outliers in the training data set.

Robust correlation coefficients that deal with bad leverage points

Consider the usual linear regression model. A well-known concern is that a bad leverage point, which is a type of outlier, can result in a poor fit to the bulk of the data, even when using any one of many robust regression estimators. In terms of measuring the strength of the association, bad leverage points can mask a strong association among the bulk of the data, and bad leverage points can suggest a strong association when in fact there is, in general, a weak association. This issue can be addressed by using an analog of Pearson’s correlation that is eliminates outliers. But this approach can have a negative impact because it eliminates what are known as good leverage points. The paper suggests a class of robust measures of association that deals with this issue.

Regression: Identifying Good and Bad Leverage Points

When dealing with regression, a well known concern is that a few bad leverage points can result in a poor fit to the bulk of the data. This is the case even when using various robust estimators, which is known as contamination bias. Currently, a relatively e ective method for detecting bad leverage points is based in part on the least median of squares regression estimator. This note suggests a modification of this method that is better able to detect bad leverage points. The modification also provides a substantially better technique for dealing with contamination bias.

Robust Variable Selection Based on Schwarz Information Criterion for Linear Regression Models

Schwarz information criterion (SIC) is a popular tool to select the best variables in regression data sets. However, SIC defined using an unbounded estimator (Least Squares (LS)) which is very sensitive to the presence of outlying observations, especially bad leverage points. Thus, robust variable selection based on SIC for linear regression models is in need. This paper study the robust properties of SIC derives its influence function and proposes robust SIC based on the MM-estimation scale, aim to produce criterion which is effective in selecting accurate models in the presence of vertical outliers and high leverage points. The advantages of the proposed robust SIC is demonstrated through simulation study and analysis of a real data set.

Detection of Influential Observations in Spatial Regression Model Based on Outliers and Bad Leverage Classification

Influential observations (IOs), which are outliers in the x direction, y direction or both, remain a problem in the classical regression model fitting. Spatial regression models have a peculiar kind of outliers because they are local in nature. Spatial regression models are also not free from the effect of influential observations. Researchers have adapted some classical regression techniques to spatial models and obtained satisfactory results. However, masking or/and swamping remains a stumbling block for such methods. In this article, we obtain a measure of spatial Studentized prediction residuals that incorporate spatial information on the dependent variable and the residuals. We propose a robust spatial diagnostic plot to classify observations into regular observations, vertical outliers, good and bad leverage points using a classification based on spatial Studentized prediction residuals and spatial diagnostic potentials, which we refer to as ISRs−Posi and ESRs−Posi. Observations that fall into the vertical outliers and bad leverage points categories are referred to as IOs. Representations of some classical regression measures of diagnostic in general spatial models are presented. The commonly used diagnostic measure in spatial diagnostics, the Cook’s distance, is compared to some robust methods, Hi2 (using robust and non-robust measures), and our proposed ISRs−Posi and ESRs−Posi plots. Results of our simulation study and applications to real data showed that the Cook’s distance, non-robust Hsi12 and robust Hsi22 were not very successful in detecting IOs. The Hsi12 suffered from the masking effect, and the robust Hsi22 suffered from swamping in general spatial models. Interestingly, the results showed that the proposed ESRs−Posi plot, followed by the ISRs−Posi plot, was very successful in classifying observations into the correct groups, hence correctly detecting the real IOs.

Outliers in Semi-Parametric Estimation of Treatment Effects

Outliers can be particularly hard to detect, creating bias and inconsistency in the semi-parametric estimates. In this paper, we use Monte Carlo simulations to demonstrate that semi-parametric methods, such as matching, are biased in the presence of outliers. Bad and good leverage point outliers are considered. Bias arises in the case of bad leverage points because they completely change the distribution of the metrics used to define counterfactuals; good leverage points, on the other hand, increase the chance of breaking the common support condition and distort the balance of the covariates, which may push practitioners to misspecify the propensity score or the distance measures. We provide some clues to identify and correct for the effects of outliers following a reweighting strategy in the spirit of the Stahel-Donoho (SD) multivariate estimator of scale and location, and the S-estimator of multivariate location (Smultiv). An application of this strategy to experimental data is also implemented.

Simple and Fast Generalized - M (GM) Estimator and Its Application to Real Data Set

It is now evident that some robust methods such as MM-estimator do not address the concept of bounded influence function, which means that their estimates still be affected by outliers in the X directions or high leverage points (HLPs), even though they have high efficiency and high breakdown point (BDP). The Generalized M(GM) estimator, such as the GM6 estimator is put forward with the main aim of making a bound for the influence of HLPs by some weight function. The limitation of GM6 is that it gives lower weight to both bad leverage points (BLPs) and good leverage points (GLPs) which make its efficiency decreases when more GLPs are present in a data set. Moreover, the GM6 takes longer computational time. In this paper, we develop a new version of GM-estimator which is based on simple and fast algorithm. The attractive feature of this method is that it only downs weights BLPs and vertical outliers (VOs) and increases its efficiency. The merit of our proposed GM estimator is studied by simulation study and well-known aircraft data set.

Kernel Partial Least Square Regression with High Resistance to Multiple Outliers and Bad Leverage Points on Near-Infrared Spectral Data Analysis

Multivariate statistical analysis such as partial least square regression (PLSR) is the common data processing technique used to handle high-dimensional data space on near-infrared (NIR) spectral datasets. The PLSR is useful to tackle the multicollinearity and heteroscedasticity problem that can be commonly found in such data space. With the problem of the nonlinear structure in the original input space, the use of the classical PLSR model might not be appropriate. In addition, the contamination of multiple outliers and high leverage points (HLPs) in the dataset could further damage the model. Generally, HLPs contain both good leverage points (GLPs) and bad leverage points (BLPs); therefore, in this case, removing the BLPs seems relevant since it has a significant impact on the parameter estimates and can slow down the convergence process. On the other hand, the GLPs provide a good efficiency in the model calibration process; thus, they should not be eliminated. In this study, robust alternatives to the existing kernel partial least square (KPLS) regression, which are called the kernel partial robust GM6-estimator (KPRGM6) regression and the kernel partial robust modified GM6-estimator (KPRMGM6) regression are introduced. The nonlinear solution on PLSR was handled through kernel-based learning by nonlinearly projecting the original input data matrix into a high-dimensional feature mapping that corresponded to the reproducing kernel Hilbert spaces (RKHS). To increase the robustness, the improvements on GM6 estimators are presented with the nonlinear PLSR. Based on the investigation using several artificial dataset scenarios from Monte Carlo simulations and two sets from the near-infrared (NIR) spectral dataset, the proposed robust KPRMGM6 is found to be superior to the robust KPRGM6 and non-robust KPLS.

An Efficient Estimation and Classification Methods for High Dimensional Data Using Robust Iteratively Reweighted SIMPLS Algorithm Based on nu-Support Vector Regression

The statistically inspired modification of the partial least squares (SIMPLS) is the most commonly used algorithm to solve a partial least squares regression problem when the number of explanatory variables ( p) is larger than the sample size ( n). Nonetheless, in the presence of irregular points (outliers), this method is no longer efficient. Therefore, the robust iteratively reweighted SIMPLS (RWSIMPLS), which is an improvement of the SIMPLS algorithm, is put forward to remedy this problem. However, the RWSIMPLS is still not very efficient with regard to its parameter estimations and outlier diagnostics. It also suffers from long computational times. This paper proposes a new robust SIMPLS that incorporates a new weight function constructed from nu-Support Vector Regression in its establishment. We call this method the robust iteratively reweighted SIMPLS based on nu-Support Vector Regression, denoted as SVR-RWSIMPLS. To avoid misclassification of observations, a new diagnostic plot is proposed to classify observations into regular observations, vertical outliers, good (GLPs) and bad leverage points (BLPs). The numerical results clearly indicate that the SVR-RWSIMPLS is more efficient, more robust and has less computational running times than the RWSIMPLS when multiple leverage points and vertical outliers exist. The proposed diagnostic plot is also very successful in classifying observations into correct groups.

Vulnerability of the Largest Normalized Residual Statistical Test to Leverage Points

The largest normalized residual statistical test (LNRT) is claimed to have the ability to detect noncritical single or multiple interacting but nonconforming bad data. However, we show in this letter that this claim does not always hold true for bad leverage points. Specifically, we prove that while a single noncritical and nonleverage measurement that is corrupted by a sufficiently large gross error is likely to be detected, the same measurement in a position of leverage and corrupted by the same gross error is unlikely to be detected. Furthermore, we show that multiple noncritical bad leverage points that are topologically adjacent to each other naturally form one or several groups of interacting bad data, which may cause the LNRT to fail. Since leverage points are common in actual power systems, the LNRT is doomed to frequently fail to detect bad data. Simulation results carried out on IEEE 30-bus system validate our theoretical results.

Sparse State Recovery Versus Generalized Maximum-Likelihood Estimator of a Power System

This letter shows that the sparse state recovery optimization method is equivalent to the well-known Huber M-estimator, and then justifies its robustness to bad data. We derive the total influence functions of the Huber M-estimator and the generalized maximum-likelihood (GM)-estimator, and give a formal proof that the Huber M-estimator is vulnerable to bad leverage points while the GM-estimator can handle them. Numerical results carried out on various IEEE systems validate our theoretical results.

Robust Clustering in Regression Analysis via the Contaminated Gaussian Cluster-Weighted Model

The Gaussian cluster-weighted model (CWM) is a mixture of regression models with random covariates that allows for flexible clustering of a random vector composed of response variables and covariates. In each mixture component, a Gaussian distribution is adopted for both the covariates and the responses given the covariates. To make the approach robust with respect to the presence of mildly atypical observations, the contaminated Gaussian CWM is introduced. In addition to the parameters of the Gaussian CWM, each mixture component has a parameter controlling the proportion of outliers, one controlling the proportion of leverage points, one specifying the degree of contamination with respect to the response variables, and another specifying the degree of contamination with respect to the covariates. Crucially, these parameters do not have to be specified a priori, adding flexibility to the approach. Furthermore, once the model is estimated and the observations are assigned to the components, a finer intra-group classification in typical points, (mild) outliers, good leverage points, and bad leverage points—concepts of primary importance in robust regression analysis—can be directly obtained. Relations with other mixture-based contaminated models are analyzed, identifiability conditions are provided, an expectation-conditional maximization algorithm is outlined for parameter estimation, and various implementation and operational issues are discussed. Properties of the estimators of the regression coefficients are evaluated through Monte Carlo experiments and compared with other procedures. A sensitivity study is also conducted based on a real data set.

Outliers in Semi-Parametric Estimation of Treatment Effects

Average treatment effects estimands can present significant bias under the presence of outliers. Moreover, outliers can be particularly hard to detect, creating bias and inconsistency in the semi-parametric ATE estimands. In this paper, we use Monte Carlo simulations to demonstrate that semi-parametric methods, such as matching, are biased in the presence of outliers. Bad and good leverage points outliers are considered. The bias arises because bad leverage points completely change the distribution of the metrics used to define counterfactuals. Whereas good leverage points increase the chance of breaking the common support condition and distort the balance of the covariates and which may push practitioners to misspecify the propensity score. We provide some clues to diagnose the presence of outliers and propose a reweighting estimator that is robust against outliers based on the Stahel-Donoho multivariate estimator of scale and location. An application of this estimator to LaLonde (1986) data allows us to explain the Dehejia and Wahba (2002) and Smith and Todd (2005) debate on the inability of matching estimators to deal with the evaluation problem.

A high breakdown, high efficiency and bounded influence modified GM estimator based on support vector regression

ABSTRACTRegression analysis aims to estimate the approximate relationship between the response variable and the explanatory variables. This can be done using classical methods such as ordinary least squares. Unfortunately, these methods are very sensitive to anomalous points, often called outliers, in the data set. The main contribution of this article is to propose a new version of the Generalized M-estimator that provides good resistance against vertical outliers and bad leverage points. The advantage of this method over the existing methods is that it does not minimize the weight of the good leverage points, and this increases the efficiency of this estimator. To achieve this goal, the fixed parameters support vector regression technique is used to identify and minimize the weight of outliers and bad leverage points. The effectiveness of the proposed estimator is investigated using real and simulated data sets.

Robust estimation of mixtures of regressions with random covariates, via trimming and constraints

A robust estimator for a wide family of mixtures of linear regression is presented. Robustness is based on the joint adoption of the cluster weighted model and of an estimator based on trimming and restrictions. The selected model provides the conditional distribution of the response for each group, as in mixtures of regression, and further supplies local distributions for the explanatory variables. A novel version of the restrictions has been devised, under this model, for separately controlling the two sources of variability identified in it. This proposal avoids singularities in the log-likelihood, caused by approximate local collinearity in the explanatory variables or local exact fits in regressions, and reduces the occurrence of spurious local maximizers. In a natural way, due to the interaction between the model and the estimator, the procedure is able to resist the harmful influence of bad leverage points along the estimation of the mixture of regressions, which is still an open issue in the literature. The given methodology defines a well-posed statistical problem, whose estimator exists and is consistent to the corresponding solution of the population optimum, under widely general conditions. A feasible EM algorithm has also been provided to obtain the corresponding estimation. Many simulated examples and two real datasets have been chosen to show the ability of the procedure, on the one hand, to detect anomalous data, and, on the other hand, to identify the real cluster regressions without the influence of contamination.

A New Robust Diagnostic Plot for Classifying Good and Bad High Leverage Points in a Multiple Linear Regression Model

Identification of high leverage point is crucial because it is responsible for inaccurate prediction and invalid inferential statement as it has a larger impact on the computed values of various estimates. It is essential to classify the high leverage points into good and bad leverage points because only the bad leverage points have an undue effect on the parameter estimates. It is now evident that when a group of high leverage points is present in a data set, the existing robust diagnostic plot fails to classify them correctly. This problem is due to the masking and swamping effects. In this paper, we propose a new robust diagnostic plot to correctly classify the good and bad leverage points by reducing both masking and swamping effects. The formulation of the proposed plot is based on the Modified Generalized Studentized Residuals. We investigate the performance of our proposed method by employing a Monte Carlo simulation study and some well-known data sets. The results indicate that the proposed method is able to improve the rate of detection of bad leverage points and also to reduce swamping and masking effects.

Heteroscedastic Global Tests that the Regression Parameters for Two or More Independent Groups are Identical

Simulation results are reported on methods that allow both within group and between group heteroscedasticity when testing the hypothesis that independent groups have identical regression parameters. The methods are based on a combination of extant techniques, but their finite-sample properties have not been studied. Included are results on the impact of removing all leverage points or just bad leverage points. The method used to identify leverage points can be important and can improve control over the Type I error probability. Results are illustrated using data from the Well Elderly II study.

Constrained Robust Estimation of Power System State Variables and Transformer Tap Positions Under Erroneous Zero-Injections

This paper presents an equally constrained robust estimator of both the state and the transformer tap positions of a power system able to withstand all types of outliers, including bad leverage points and erroneous zero-injections. The statistical robustness of the estimator stems from the application of the Schweppe-type Huber GM estimator (SHGM) while its numerical robustness originates from the use of an orthogonal iteratively re-weighted least-squares algorithm together with the Van Loan's method for processing the equality constraints. The good performance of the new estimator, termed EC-SHGM estimator for short, is demonstrated on a small test system and on the Brazilian Southern power system with increasing size ranging from 139 buses to 1916 buses. It is shown that it exhibits superior convergence properties in all tested cases while the WLS method may suffer from numerical instabilities or even divergence problems when large weights are assigned to zero power injections modeling false information.

MODEL SELECTION VIA ROBUST VERSION OF R-SQUARED

R-squared (R2) is a popular method for variable selection in linear regression models. R2 based on Least Squares (LS) regression minimizes the sum of the squared residuals; LS is sensitive to outlier observation. Alternative criterion based on M-estimators, which is less sensitive to outlying observation has been proposed. In this study explicit expression for such criterion is obtained when the Least Trimmed Squares (LTS) estimator is used. The influence function of R2 is also discussed. In our simulation study, the performance of proposed criterion is compared to the existing criteria based on M-estimators (R2M) and to the classical non-robust based on least squares estimators (R2LS). We observe that the proposed (R2LTS) selects more appropriate models in the case of bad leverage points (outliers in the X-direction) are present.