Least Median Of Squares Estimator Research Articles

MENGATASI PENCILAN PADA PEMODELAN REGRESI LINEAR BERGANDA DENGAN METODE REGRESI ROBUST PENAKSIR LMS

Ordinary Least Squares (OLS) is frequent used method for estimating parameters. OLS estimator is not a robust regression procedure for the presence of outliers, so the estimate becomes inappropriate. Least Median of Squares (LMS) is one of a robust estimator for the presence of outliers and has a high breakdown value. LMS estimate parameters by minimizing the median of squared residuals. Least Median of Squares (LMS) The purpose of this study is geting a regression equation that better than the regression equation before using OLS for the data that having outlier. For the first step, checking if there is outlier at data and then searching regression equation with LMS method. In this study used data stackloss and from estimation parameter of this data, LMS estimator showed better results compared to the OLS estimator because the regression equation from LMS method have smaller value of Mean Absolute Percentage Error (MAPE).

Models Where the Least Trimmed Squares and Least Median of Squares Estimators Are Maximum Likelihood

The Least Trimmed Squares (LTS) and Least Median of Squares (LMS) estimators are popular robust regression estimators. The idea behind the estimators is to find, for a given h, a sub-sample of h 'good' observations among n observations and estimate the regression on that sub-sample. We find models, based on the normal or the uniform distribution respectively, in which these estimators are maximum likelihood. We provide an asymptotic theory for the location-scale case in those models. The LTS estimator is found to be h1/2 consistent and asymptotically standard normal. The LMS estimator is found to be h consistent and asymptotically Laplace.

Robust circle reconstruction with the Riemann fit

Finding and fitting circles from a set of points is a frequent problem in the data analysis of high-energy physics experiments. In a tracker immersed in a homogeneous magnetic field, tracks are close to perfect circles if projected to the bending plane. In a ring-imaging Cherenkov (RICH) detector, circles of photons around the crossing point of charged particles have to be found and their radii estimated. In both cases, non-negligible background may be present that tends to complicate the pattern recognition and to bias the circle fit. In this contribution we present a robust circle fit based on a modified Riemann fit that removes or significantly reduces the effect of background points. As in the standard Riemann fit, the measured points are projected to the Riemann sphere or paraboloid, and a plane is fitted to the projected points. The fit is made robust by replacing the usual least-squares regression by a least median of squares (LMS) regression. Because of the high breakdown point of the LMS estimator, the fit is insensitive to background points. The LMS plane is used to initialize the weights of an M-estimator that refits the plane in order to suppress eventual remaining outliers and to obtain the final circle parameters. The method is demonstrated on three sets of artificial data: points on a circle plus a comparable number of background points; points on two overlapping circles with additional background; and points obtained by the simulation of tracks in a drift chamber with mirror points and additional background. The results show high circle finding efficiency and small contamination of the final fitted circles.

Regular, median and Huber cross‐validation: A computational comparison

AbstractWe present a new technique for comparing models using a median form of cross‐validation and least median of squares estimation (MCV‐LMS). Rather than minimizing the sums of squares of residual errors, we minimize the median of the squared residual errors. We compare this with a robustified form of cross‐validation using the Huber loss function and robust coefficient estimators (HCV). Through extensive simulations we find that for linear models MCV‐LMS outperforms HCV for data that is representative of the data generator when the tails of the noise distribution are heavy enough and asymmetric enough. We also find that MCV‐LMS is often better able to detect the presence of small terms. Otherwise, HCV typically outperforms MCV‐LMS for ‘good’ data. MCV‐LMS also outperforms HCV in the presence of enough severe outliers.One of MCV and HCV also generally gives better model selection for linear models than the conventional version of cross‐validation with least squares estimators (CV‐LS) when the tails of the noise distribution are heavy or asymmetric or when the coefficients are small and the data is representative. CV‐LS only performs well when the tails of the error distribution are light and symmetric and the coefficients are large relative to the noise variance. Outside of these contexts and the contexts noted above, HCV outperforms CV‐LS and MCV‐LMS.We illustrate CV‐LS, HVC, and MCV‐LMS via numerous simulations to map out when each does best on representative data and then apply all three to a real dataset from econometrics that includes outliers.

Asymptotics of Bayesian median loss estimation

We establish the consistency, asymptotic normality, and efficiency for estimators derived by minimizing the median of a loss function in a Bayesian context. We contrast this procedure with the behavior of two Frequentist procedures, the least median of squares (LMS) and the least trimmed squares (LTS) estimators, in regression problems. The LMS estimator is the Frequentist version of our estimator, and the LTS estimator approaches a median-based estimator as the trimming approaches 50% on each side. We argue that the Bayesian median-based method is a good tradeoff between the two Frequentist estimators.

Least median of squares estimation by optimization heuristics with an application to the CAPM and a multi-factor model

For estimating the parameters of models for financial market data, the use of robust techniques is of particular interest. Conditional forecasts, based on the capital asset pricing model, and a factor model are considered. It is proposed to consider least median of squares estimators as one possible alternative to ordinary least squares. Given the complexity of the objective function for the least median of squares estimator, the estimates are obtained by means of optimization heuristics. The performance of two heuristics is compared, namely differential evolution and threshold accepting. It is shown that these methods are well suited to obtain least median of squares estimators for real world problems. Furthermore, it is analyzed to what extent parameter estimates and conditional forecasts differ between the two estimators. The empirical analysis considers daily and monthly data on some stocks from the Dow Jones Industrial Average Index.

On the Least Median Square Problem

We consider the exact and approximate computational complexity of the multivariate least median-of-squares (LMS) linear regression estimator. The LMS estimator is among the most widely used robust linear statistical estimators. Given a set of n points in \({\Bbb R}^d\) and a parameter k, the problem is equivalent to computing the narrowest slab bounded by two parallel hyperplanes that contains k of the points. We present algorithms for the exact and approximate versions of the multivariate LMS problem. We also provide nearly matching lower bounds for these problems. These lower bounds hold under the assumptions that k is Ω(n) and that deciding whether n given points in \({\Bbb R}^d\) are affinely non-degenerate requires Ω(nd) time.

Georegistration of Landsat data via robust matching of multiresolution features

The goal of the project described in this paper is to build a prototype of an operational system, which will provide registration within subpixel accuracy of multitemporal Landsat data, acquired by either Landsat-5 or Landsat-7 Thematic Mapper instruments. Integrated within an automated mass processing system for Landsat data, the input to our registration system consists of scenes that have been geometrically and radiometrically corrected, as well as preprocessed for detection of clouds and cloud shadows. Such preprocessed scenes are then georegistered relative to a database of Landsat chips. This paper describes the entire registration process, including the use of landmark chips, feature extraction performed by an overcomplete wavelet representation, and feature matching using statistically robust techniques. Knowing the approximate longitudes and latitudes or the UTM coordinates of the four corners of each incoming scene, a subset of the chips that represent landmarks included in the scene are selected to perform the registration. For each of these selected landmark chips, a corresponding window is extracted from the incoming scene, and each chip-window pair is registered using a robust wavelet feature-matching methodology. Based on the transformations from the chip-window pairs, a global transformation is then computed for the entire scene using a variant of a robust least median of squares estimator. Empirical results of this registration process, which provided subpixel accuracy for several multitemporal scenes from different study areas, are presented and discussed.

ASYMPTOTIC THEORY FOR SOME HIGH BREAKDOWN POINT ESTIMATORS

High breakdown point estimators in regression are robust against gross contamination in the regressors and also in the errors; the least median of squares (LMS) estimator has the additional property of packing the majority of the sample most tightly around the estimated regression hyperplane in terms of absolute deviations of the residuals and thus is helpful in identifying outliers. Asymptotics for a class of high breakdown point smoothed LMS estimators are derived here under a variety of conditions that allow for time series applications; joint limit processes for several smoothed estimators are examined. The limit process for the LMS estimator is represented via a generalized Gaussian process that defines the generalized derivative of the Wiener process.

Robust Fuzzy Linear Regression And Application For Contact Identification

Abstract Linear regression has been used for a long time in various applications, with linear least squares as the best known tool. It is also known in probability / statistics theory that this method is particularly sensitive to outliers. Hence, statisticians introduced robust statistical tools that allow robustness and efficiency to overcome the effects of outlier. On the other hand, Tanaka and Hayashi provide basic ideas for fuzzy linear regression when data are rather ill-known and given in terms of fuzzy sets, even if the robustness of the method in the presence of outliers still is poor and over-estimated. This paper attempts to provide a hybrid approach by combining both robust statistical tools and the fuzzy approach. Particularly, the least median of squares estimator and the least trimmed squares estimator have been considered. The method is then tested in a robotics application where aforce-controlled contact situation is assessed.

Relationships between maximum depth and projection regression estimates

This article deals with the relationships between regression estimates based on projections (Ann. Statist. 21 (1993) 965) and on maximum depth (J. Amer. Statist. Assoc. 94 (1999) 388), which are shown to share the same basic idea. The maximum asymptotic bias of the latter is derived. A measure of residual smallness is defined, which turns out to be equivalent to depth. Two approximate algorithms based on subsampling are proposed to compute the maximum depth estimate: one based on the original definition, and the other on the new criterion. Finally, a Monte Carlo study is performed to compare the least median of squares estimate, the projection estimate, and the two approximate versions of the maximum depth estimate. It is seen that the latter have rather different behaviors, and the best overall performance corresponds to the projection estimate and one of the versions of the depth estimate.

Improving bias-robustness of regression estimates through projections

We define a robust procedure to “correct” a regression estimate β ̂ 0 along the directions in predictor space where the fit is worse. When β ̂ 0 is the least median of squares estimate, the “corrected estimate” has a smaller maximum asymptotic bias under contamination, and a much better finite-sample behavior than β ̂ 0.

Finite sample stability properties of the least median of squares estimator

This article examines the stability properties of the least median of squares (LMS) estimate. Attention is focussed on LMS since it is arguably the most widely used high breakdown regression estimate. The differing roles of breakdown point and influence function in producing estimates which are stable to changes in the data are discussed. Simulations and real examples are used to illustrate the extent to which the LMS estimate can change when small changes are made to centrally located data. It is shown that this instability is a consequence of the fact that the LMS estimate has an influence function which is unbounded to the effects of centrally located x's and is not merely a consequence of the exact fit property and the curse of dimensionality, as argued by Rousseeuw (1994).

Asymptotic robustness of least median of squares for autoregressions with additive outliers

This paper considers the robustness properties in the time series context of the least median of squares (LMS) estimator. The influence function of the LMS estimator is derived under additive outlier contamination. This influence function is redescending and bounded for fixed values of the AR parameters. The gross-error sensitivity, however, is an unbounded function of the AR parameters. In order to asses the global robustness behavior of the LMS estimator, we consider several notions of breakdown. The breakdown points of the LMS estimator depend on the value of the underlying AR parameter. Generally, the breakdown point is below one half for high values of the AR parameter. The bias curves of the LMS estimator reveal, however, that the magnitude of outliers has to be considerable in order to cause breakdown.

Outlier removal and discontinuity preserving smoothing of range data

Algorithms for the segmentation and description of range images are very sensitive to errors in the source data caused by noise processes in the optoelectronic sensing, and outliers caused by incorrect signal detection, for example false peaks in an active laser triangulation system. The authors present an approach to range data processing designed to reconstruct the underlying shape of the surfaces in the scene, yet preserve the discontinuities between them. The approach has two stages, first outlier removal by a lower complexity variation of the least median of squares estimator, and second, robust smoothing by anisotropic diffusion. To evaluate the proposed methods, the authors quantify the improvement in depth, normal and curvature estimation, and show how preprocessing improves surface patch segmentation and classification.

Identification of multiple interacting bad data via power system decomposition

This paper presents a new, highly robust bad data identification algorithm for electric power system state estimation. A system decomposition scheme is coupled with the least median of squares estimator to allow identification of multiple interacting bad data even in cases of conforming errors. The algorithm is inherently resistant to bad measurements in positions of leverage, makes no a priori measurement error probability distribution assumptions, and is applicable in a real-time environment.

Consistency of the least median of squares estimator in nonlinear regression

Recent research by Sakata and White (1995) presents the consistency and asymptotic normality of S-estimators in nonlinear regression. It is well known from research in linear regression that it is important to use a consistent high breakdown estimator as an initial estimate when computing an S-estimate. This paper presents the proof of the weak consistency of the least median of squares estimator in a nonlinear regression setting, thus suggesting that it is a reasonable choice for the starting value for computing S-estimates in nonlinear regression.

Least median of weighted squares in logistic regression with large strata

One aim of robust regression is to find estimators with high finite sample breakdown points. Although various robust estimators have been proposed in logistic regression models, their breakdown points are not yet known. Here it is shown for logistic regression models with binary data that there is no estimator with a high finite sample breakdown point, provided the estimator has to fulfill a weak condition. In logistic regression models with large strata however, a modification of Rousseeuw's least median of squares estimator is shown to have a finite sample breakdown point of approximately ½.

Aspects of Robust Linear Regression

Section 1 of the paper contains a general discussion of robustness. In Section 2 the influence function of the Hampel-Rousseeuw least median of squares estimator is derived. Linearly invariant weak metrics are constructed in Section 3. It is shown in Section 4 that $S$-estimators satisfy an exact Holder condition of order 1/2 at models with normal errors. In Section 5 the breakdown points of the Hampel-Krasker dispersion and regression functionals are shown to be 0. The exact breakdown point of the Krasker-Welsch dispersion functional is obtained as well as bounds for the corresponding regression functional. Section 6 contains the construction of a linearly equivariant, high breakdown and locally Lipschitz dispersion functional for any design distribution. In Section 7 it is shown that there is no inherent contradiction between efficiency and a high breakdown point. Section 8 contains a linearly equivariant, high breakdown regression functional which is Lipschitz continuous at models with normal errors.

Computing the Exact Least Median of Squares Estimate and Stability Diagnostics in Multiple Linear Regression

The difficulty in computing the least median of squares (LMS) estimate in multiple linear regression is due to the nondifferentiability and many local minima of the objective function. Several approximate, but not exact, algorithms have been suggested. This paper presents a method for computing the exact value of the LMS estimate in multiple linear regression. The LMS estimate is a special case of the least quantile of squares (LQS) estimate, which minimizes the qth smallest squared residual for a given data set. For LMS, $q = [n/2] + [(p + 1)/2]$ where $[ \, ]$ is the greatest integer function, n is the sample size, and p is the number of columns in the X matrix. The algorithm can compute a range of exact LQS estimates in multiple linear regression by considering $\left( {\begin{array}{*{20}c} n \\ {p + 1} \\ \end{array} } \right)$ possible $\theta $ values. It is based on the fact that each LQS estimate is the Chebyshev (or minimax) fit to some q element subset of the data. This yields a surprisingly easy algorithm for computing the exact LQS estimates. These and other estimates are used to study the stability of the LMS estimate in several examples.