Vertical Outliers Research Articles

Leverages, Outliers and the Performance of Robust Regression Estimators

This study compared the performance of some robust regression methods and the Ordinary Least Squares Estimator (OLSE). The estimators were compared using varied levels of leverages and vertical outliers in the predictors and the dependent variables. An anthropometric dataset on total body fat with height, Body Mass Index (BMI), Triceps Skin-fold(TS), and arm fat as percent composition of the body (parmfat), as the predictors. The e ects of outliers and leverages on the estimators, were investigated at (5% leverages and 10% vertical outliers, 5% leverages with 15% vertical outliers). The criteria for the comparison: coe cients, Root Mean Square Error (RMSE), Relative E ciencies (RE), coe cients of determination (R-squared) and power of the test. The ndings from this study revealed that, OLSE was a ected by both outliers and leverages whilst Huber Maximum likelihood Estimator (HME) was a ected by leverages. The Least Trimmed Squares Estimator (LTSE) was slightly a ected by high perturbations of outliers and leverages. The study also showed that Modi ed Maximum likelihood Estimator (MME) and S Estimator (SE) were robust to all levels of outliers and leverage perturbations. Therefore leverages and outliers in datasets do a ect the post hoc power analysis of the methods which cannot resist them.

Sparse Partial Robust M Regression

Sparse partial robust M regression is introduced as a new regression method. It is the first dimension reduction and regression algorithm that yields estimates with a partial least squares alike interpretability that are sparse and robust with respect to both vertical outliers and leverage points. A simulation study underpins these claims. Real data examples illustrate the validity of the approach.

Sparse partial robust M regression

Sparse partial robust M regression is introduced as a new regression method. It is the first dimension reduction and regression algorithm that yields estimates with a partial least squares like interpretability that are sparse and robust with respect to both vertical outliers and leverage points. A simulation study underpins these claims. Real data examples illustrate the validity of the approach.

Robust nonnegative garrote variable selection in linear regression

Robust selection of variables in a linear regression model is investigated. Many variable selection methods are available, but very few methods are designed to avoid sensitivity to vertical outliers as well as to leverage points. The nonnegative garrote method is a powerful variable selection method, developed originally for linear regression but recently successfully extended to more complex regression models. The method has good performances and its theoretical properties have been established. The aim is to robustify the nonnegative garrote method for linear regression as to make it robust to vertical outliers and leverage points. Several approaches are discussed, and recommendations towards a final good performing robust nonnegative garrote method are given. The proposed method is evaluated via a simulation study that also includes a comparison with existing methods. The method performs very well, and often outperforms existing methods. A real data application illustrates the use of the method in practice.

Least median of squares regression and minimum volume ellipsoid estimator for outliers detection in housing appraisal

In the real estate sector, regression analysis is the most used method for interpretative and predictive purposes. However, the presence of outliers in the estimative sample can lead to ordinary last squared regression models that do not represent the investigated market phenomenon, with the consequence of producing unreliable assessments. In the present research, the issues of the identification and the removal of outliers are discussed. The outliers identified by the least median of squares (LMS) regression and the minimum volume ellipsoid (MVE) estimator are compared in order to test the coincidence or the diversity. A complete diagnosis of the data of the initial estimative sample is carried out, combining the robust residuals obtained with LMS and the robust distances obtained with MVE. The data are classified into regular observations, vertical outliers, good leverage points and bad leverage points, and cases to delete and those to keep in the sample are identified.

A new GM-estimate with high breakdown point

The breakdown point of GM-estimators does not exceed 1/(p+1) where p is dimension of explanatory variables in linear regression. One proposes a new version of the GM-estimation with a high breakdown point (HBGM) to provide high resistance against leverage points and vertical outliers. This paper presents a technique aimed at routinely normalized and robustified Euclidean distance among data points for finding leverage points and gross errors in the x- and y-directions, respectively. In addition, a graphical visualization is simultaneously used to display which points are leverage and gross error. Finally, weights of flagged data points will be decreased to a certain value before applying M-estimator. Since robustification and normalization procedures are completely based on the median estimator with the highest breakdown point, the proposed method has a conditional breakdown point of 50 % theoretically. The technique was tested with simulated data and also real data set containing the series of landslide deformation. Tests were performed for linear regression models including different scenarios. Consequently, the experimental results showed that the proposed method reaches up to a 50 % of breakdown point. Morever, the HBGM method is less time- consuming in parameter estimation.

Robust Regression When the True R2 is Mediocre

It is well known that several high-breakdown procedures can produce robust estimates of linear-regression coefficients despite substantial contamination in the data. In practice, however, the estimators’ robustness tends to be affected by the value of R2 in the uncontaminated part of the sample: when R2 is mediocre, commonly-recommended high-breakdown estimators may retain a substantial bias due to the contaminated data. This paper reports several simulations of simple and multiple regression where vertical outliers and bad-leverage observations affect 25% of the data and R2 is mediocre. One high-breakdown estimator appears to have small bias, very good coverage of a nominal 90% confidence interval, and precision that improves when the sample size is increased. The simulations are supplemented by applications to four real data sets.

로버스트추정에 바탕을 둔 주성분로지스틱회귀

로지스틱회귀분석은 고객관계관리를 위한 데이터마이닝 분야에서 많이 사용되는 기법인데, 이 분야의 모형설정 과정에서는 연관성이 매우 높은 설명변수들이 모형에 함께 포함되어 다중공선성의 문제를 유발하며, 더욱이 회귀자료에 이상점들이 포함되면 최우추정량은 심각한 결함을 갖게 된다. 두 가지 문제점을 동시에 해결하기 위하여 로버스트주성분로지스틱회귀를 적용할 수 있는데, 본 논문에서는 주성분의 선정기준을 결정하는 모형을 개발하고, 주성분모형에서의 추정치에 미치는 이상점의 영향을 축소하기 위한 로버스트추정법을 제안하였다. 제안된 추정법은 다중공선성과 이상점이 유발하는 문제들을 적절히 해결해 준다는 사실이 모의실험을 통하여 확인되었다. Logistic regression is widely used as a datamining technique for the customer relationship management. The maximum likelihood estimator has highly inflated variance when multicollinearity exists among the regressors, and it is not robust against outliers. Thus we propose the robust principal components logistic regression to deal with both multicollinearity and outlier problem. A procedure is suggested for the selection of principal components, which is based on the condition index. When a condition index is larger than the cutoff value obtained from the model constructed on the basis of the conjoint analysis, the corresponding principal component is removed from the logistic model. In addition, we employ an algorithm for the robust estimation, which strives to dampen the effect of outliers by applying the appropriate weights and factors to the leverage points and vertical outliers identified by the V-mask type criterion. The Monte Carlo simulation results indicate that the proposed procedure yields higher rate of correct classification than the existing method.

Beware of ‘Good’ Outliers and Overoptimistic Conclusions*

AbstractThe main goal of this paper is to warn practitioners of the danger of neglecting outliers in regression analysis, in particular, good leverage points (i.e. points lying close to the regression hyperplane but outlying in the x‐dimension). While the types of outliers which do influence regression estimates (vertical outliers and bad leverage points) have been extensively investigated, good leverage points have been largely ignored, probably because they do not affect the estimated regression parameters. However, their effect on inference is far from negligible. We propose a step‐by‐step procedure to identify and treat all types of outliers. The paper of Persson and Tabellini [American Economic Review (2004) Vol. 94, pp. 25–46] linking the degree of proportionality of an electoral system to the size of government is discussed to illustrate how the choice of a measure and the existence of atypical observations may substantially influence results.

로지스틱회귀모형의 로버스트 추정을 위한 알고리즘

로지스틱회귀에서 일반적으로 사용되는 최대우도추정법은 이상점에 대해 로버스트 하지 않다. 따라서 본 논문에서는 로지스틱회귀모형의 로버스트 추정을 위한 알고리즘을 제안하고자 한다. 이 알고리즘은 V-마스크 형태의 경계기준에 의해 나쁜 지렛점과 수직이상점을 식별하고, 식별 결과를 바탕으로 이상점의 영향력을 감소시키기 위한 효과적인 방안을 모색한다. 이상점의 영향력 감소는 가중치와 조정치를 적절히 선정함으로 가능하며, 그 결과 붕괴점이 높은 추정치를 얻게 된다. 제안된 알고리즘을 다양한 자료에 적용하여 정분류율을 측정하여 비교하였는데, 새로운 알고리즘이 최대우도추정보다 정확한 분류를 해 주는 것으로 평가되었다. The maximum likelihood estimation is not robust against outliers in the logistic regression. Thus we propose an algorithm for the robust estimation, which identifies the bad leverage points and vertical outliers by the V-mask type criterion, and then strives to dampen the effect of outliers. Our main finding is that, by an appropriate selection of weights and factors, we could obtain the logistic estimates with high breakdown point. The proposed algorithm is evaluated by means of the correct classification rate on the basis of real-life and artificial data sets. The results indicate that the proposed algorithm is superior to the maximum likelihood estimation in terms of the classification.

A diagnostic plot for regression outliers and leverage points

A new diagnostic display is proposed, which plots robust residuals versus robust distances in x-space. This has many advantages. First, robust residuals (obtained from a high-breakdown fit) are better indicators of regression outliers than are least squares residuals, which suffer from masking. For similar reasons, robust distances diagnose leverage points much more reliably than do the classical Mahalanobis distances or hat diagonals. And finally, the proposed plot enables the user to distinguish between all four types of points: regular observations, vertical outliers, good leverage points, and bad leverage points.

Unmasking Multivariate Outliers and Leverage Points

Abstract Detecting outliers in a multivariate point cloud is not trivial, especially when there are several outliers. The classical identification method does not always find them, because it is based on the sample mean and covariance matrix, which are themselves affected by the outliers. That is how the outliers get masked. To avoid the masking effect, we propose to compute distances based on very robust estimates of location and covariance. These robust distances are better suited to expose the outliers. In the case of regression data, the classical least squares approach masks outliers in a similar way. Also here, the outliers may be unmasked by using a highly robust regression method. Finally, a new display is proposed in which the robust regression residuals are plotted versus the robust distances. This plot classifies the data into regular observations, vertical outliers, good leverage points, and bad leverage points. Several examples are discussed.

Unmasking Multivariate Outliers and Leverage Points

Abstract Detecting outliers in a multivariate point cloud is not trivial, especially when there are several outliers. The classical identification method does not always find them, because it is based on the sample mean and covariance matrix, which are themselves affected by the outliers. That is how the outliers get masked. To avoid the masking effect, we propose to compute distances based on very robust estimates of location and covariance. These robust distances are better suited to expose the outliers. In the case of regression data, the classical least squares approach masks outliers in a similar way. Also here, the outliers may be unmasked by using a highly robust regression method. Finally, a new display is proposed in which the robust regression residuals are plotted versus the robust distances. This plot classifies the data into regular observations, vertical outliers, good leverage points, and bad leverage points. Several examples are discussed.