Huber M-estimator Research Articles

Duality in robust linear regression using Huber's M-estimator

The robust linear regression problem using Huber's piecewise-quadratic M-estimator function is considered. Without exception, computational algorithms for this problem have been primal in nature. In this note, a dual formulation of this problem is derived using Lagrangean duality. It is shown that the dual problem is a strictly convex separable quadratic minimization problem with linear equality and box constraints. Furthermore, the primal solution (Huber's M-estimate) is obtained as the optimal values of the Lagrange multipliers associated with the dual problem. As a result, Huber's M-estimate can be computed using off-the-shelf optimization software.

Implementing nonquadratic objective functions for state estimation and bad data rejection

Using a nonquadratic objective function for network state estimation can combine several estimation and bad data rejection techniques into one algorithm: e.g. the benefits of maximum likelihood least squares estimation can be coupled with the bad data rejection properties of least absolute value estimation. For such estimators, we describe an efficient implementation, one that builds naturally on existing least squares software, that is based on an iterative Gauss-Newton solution of the KKT optimality conditions. We illustrate the behavior of a quadratic-linear and a quadratic-constant objective function on a set of test networks. The former is closely related to the Huber M-estimator. The latter shows somewhat better bad data rejection properties, perhaps because it arises from a natural model of meter failure.

Robust statistical inference: weighted likelihoods or usual m-estimation?

We review the weighted likelihood estimating equations methodology introduced by Markatou, Basu and Lindsay (1995). and Basu, Markatou and Lindsay (1995) and compare it, in the case of symmetric and asymmetric contamination, with Huber's M-estimators of location. The simulation study shows that the weighted likelihood estimating equations estimator is at least as competitive as Huber's M-estimators in the case of symmetric contamination. In the case of asymmetric contamination it may be superior than Huber's M-estimators

An Iterative Weighted Least Squares Algorithm and Simulation Study for Censored Data M-Estimates

A popular linear regression estimator for censored data is the one proposed by Buckley and James (1979). However, this estimator is not robust to outliers, which is not surprising since it is a modified version of the uncensored data least squares estimator. Lai and Ying (1994) have proposed an M-estimator for censored data that is a generalization of the Buckley- James estimator. In this paper we discuss a weighted least squares algorithm for computing these M-estimates and compare the performance of two Huber M-estimators with the Buckley-James estimator in a simulation study. We find that the Huber M-estimators perform more robustly for a broad range of censoring and error distributions.

Duality results and proximal solutions of the Huber M-estimator problem

We investigate the interest of solving the Huber M-estimator problem by a proximal approach combined with duality theory. Three different duality schemes are developed. The first one which only deals with estimator determination yields useful information on the geometrical structure of the set of optimal solutions. The second scheme links together estimator determination and outliers detection while the third one only focuses on outliers separation. We show that these three duality schemes can be solved by the partial inverse method, i.e., a special instance of the basic proximal point algorithm, which leads to very simple updating rules. This method which is always globally convergent enjoys nice stability properties and permits parallel computations.

Robust non-recursive AR speech analysis

Abstract In this paper a robust non-recursive algorithm for estimating the linear prediction (LP) parameters of autoregressive (AR) speech signal model is proposed. Starting from Huber's robust M-estimation procedure, minimizing the sum of appropriately weighted residuals, a two-step robust LP procedure (RBLP) is derived. In the first step the Huber's convex cost function is selected to give more weights to the bulk of smaller residuals, while down-weighting the small portion of large residuals, and the Newton-type algorithm is used to minimize the adopted criterion. The proposed algorithm takes into account the non-Gaussian nature of the excitation for voiced speech, being characterized by heavier tails of the underlying distribution, which generates high-intensity signal realizations named outliers. The obtained estimates are used as a new start in the weighted least-squares procedure, based on a redescending function of the prediction residuals, which has to cut off the outliers. The experiments on both synthesized and natural speech have shown that the proposed two-step RBLP gives more efficient (less variance) and less biased estimates than the conventional LP algorithms, and a one-step RBLP based on a convex cost function.

Monte Carlo experiments with robust estimation of aquifer model parameters

Automatic calibration procedures for fitting groundwater models to measured data have almost exclusively used L 2 error norms for estimation criteria. Quadratic estimation criteria result from applying the method of maximum likelihood under the assumption of jointly normal errors in the measured head data which, for the case of uncorrelated errors with constant variance, results in the ordinary least squares criterion. Quadratic estimation criteria are extremely sensitive to errors which violate the Gaussian assumption and pose an additional source of instability for the inverse problem. We consider the use of statistically robust estimation procedures to reduce these destabilizing effects. In particular, we test the robust M-estimator of Huber with a series of Monte Carlo experiments, compute the sample mean squared error, squared bias and variance of the estimated log-hydraulic conductivities for realizations of head measurements contaminated with mixed normal noise and compare these measures to their counterparts for the OLS estimator. We show that Huber's M-estimator achieves the same estimation variance as the OLS estimator with reduced need for prior information to regularize the problem.

Rank estimates in a class of semiparametric two-sample models

We consider a two-sample semiparametric model involving a real parameter θ and a nuisance parameter F which is a distribution function. This model includes the proportional hazard, proportional odds, linear transformation and Harrington-Fleming models (1982, Biometrika, 69, 533–546). We propose two types of estimates based on ranks. The first is a rank approximation to Huber's M-estimates (1981, Robust Statistics, Wiley) and the second is a Hodges-Lehmann type rank inversion estimate (1963, Ann. Math. Statist., 34, 598–611). We obtain asymptotic normality and efficiency results. The estimates are consistent and asymptotically normal generally but fully efficient only for special cases.

Heteroscedasticity and nonnormality

In this paper we consider the problem of comparing several means under heteroscedasticity and nonnormality. By combining Huber's M-estimators with the Brown-Forsythe test , several robust procedures were developed; these procedures were compared through computer simulation studies with-the Tan-Tabatabai procedure which was developed by combining Tiku's MML estimators with the Brown-Forsythe test . The numerical results indicate clearly that the Tan-Tabatabai procedure is considerably more powerful than tests based on Huber's M-estimators over a wide range of nonnormal distributions

L-Estimation for Linear Models

Abstract Linear combinations of order statistics, or L-estimators, have played an extremely important role in the development of robust methods for the one-sample problem. We suggest analogs of L-estimators for the parameters of the linear model based on the p-dimensional “regression quantiles” proposed by Koenker and Bassett (1978). A uniform, Bahadur-type asymptotic representation of regression quantiles is established, and this yields a general asymptotic theory of L-estimators for the linear model. A leading example of the proposed estimators is an analog of the trimmed mean (TRQ), which is asymptotically equivalent to the trimmed least squares estimator studied by Ruppert and Carroll (1980), but appears to be somewhat less sensitive to influential design observations. This estimator is also asymptotically equivalent to the well-known Huber M-estimator, but offers the significant advantage that it is scale invariant. We illustrate the methods by reconsidering a mid-18th century linear model analyzed b...

Finite Algorithms for Huber’sM-Estimator

Two finite algorithms for Huber's M-estimator are presented. Both proceed in a constructive manner by moving from one partition to an adjacent one. One of the algorithms, which uses the tuning constant as a continuation parameter, also has the facility to simultaneously estimate the tuning constant and scaling factor. Stable and efficient implementation of the algorithms is presented, together with numerical results.

Outlier resistance in small samples

SUMMARY This paper presents a new measure of outlier resistance based on Huber's maximum bias. After standardizing by the maximum bias of the median for the same situation, outlier resistance seems to be essentially independent of n and depends on the fraction of contamination. The advantages over alternative measures of resistance involve this relative stability with the sample size, and the ability to measure resistance to a large fraction of outliers. As an example, this method of measuring outlier resistance is applied to a comparison of a 20% trimmed mean with a Huber M-estimator.

Some robust anova procedures under heteroscedasticity and nonnormality

In this paper we consider the problem of comparing several means under heteroscedasticity and nonnormality. By combining Huber‘s M-estimators with the Brown-Forsythe test, several robust procedures were developed; these procedures were compared through computer simulation studies with the Tan-Tabatabai procedure which was developed by combining Tiku's MML estimators with the Brown-Forsythe test. The numerical results indicate clearly that the Tan-Tabatabai procedure is considerably more powerful than tests based on Huber's M-estimators over a wide range of nonnormal distributions.

On a class of stopping times for M-estimators

For a given score function ψ = ψ( x, θ), let θ n be Huber's M-estimator for an unknown population parameter θ. Under some mild smoothness assumptions it is known that n 1 2 (θ n − θ) is asymptotically normal. In this paper the stopping times τ c(m) = inf{n ≥ m: n 1 2 |θ n − θ | > c } associated with the sequence of confidence intervals for θ are investigated. A useful representation of M-estimators is derived, which is also appropriate for proving laws of the iterated logarithm and Donskertype invariance principles for ( π n ) n .

Random quotients and robust estimation

This paper compares the tail-heaviness of certain random quotients in terms of the asymptotic relative efficiences of the sample median to a large class of estimators containing the mean, trimmed mean and Huber's M-estimator. The random quotients are generalizations of the Normalllndependent distributions and include the Student's t, contaminated normal, double exponential and slash distributions.

Robust analysis of variance based upon a likelihood ratio criterion

SUMMARY Robust tests of general linear hypotheses in linear models are developed. These are likeli- hood ratio type tests in the same sense that M-estimates are maximum likelihood type estimates. Construction of the tests suggests a decomposition of the data into terms analogous to classical sums of squares, providing a robust analysis of variance. Asymptotic efficiency and robustness properties of the tests are the same as those of the M-estimates upon which they are based. Parameter estimation is usually only a first step in the analysis of data arising fromn a linear model. A classical least squares analysis often focuses upon the analysis of variance, which tests simultaneous hypotheses on large subsets of the parameters. Since the terms in a classical analysis of variance are quadratic forms in least squares estimates, one would expect that the sensitivity of the estimates to departures from normality should be inherited by the tests. In fact, for moderate to heavy tailed error distributions or in the presence of outliers, it appears that the classical F test does lose power. Calculations of relative efficiency for proce- dures proposed in this paper substantiate on theoretical grounds the possible inefficiency and lack of power of classical F tests. In this paper, Huber's M-estimates are adapted to hypothesis tests which can be termed likelihood ratio type tests. These procedures naturally generalize and bear a striking re- semblance to classical F tests. Robustness and efficiency properties of M-estimates apply directly to the proposed tests. Hence the case to be made for using likelihood ratio type tests rather than classical F tests is the same as that for using M-estimates in favour of least squares estimates: possible poor performance of the classical methods may be overcome with methods which perform well both when classical assumptions are met and when they are not. The proposed methods are natural, intuitive and as easily computed as M-estimates.

Asymptotic behavior of iterative M-estimartors for location

In this paper, we study the Huber M-estimator for location when they are computed by a class of numerical iterative procedures. This class includes the usual method of Newton-Raphson, iterated weighted least squares and iterated winsorization. We show that under mild conditions, the numerical iterative procedures converge and the resulting estimators are consistent and asymptotically normal.