Huber M-estimator Research Articles

A simulation study on classic and robust variable selection in linear regression

In linear regression analysis, outliers often have large influence in the variable selection process. The aim of this study is to select the subsets of independent variables, which explain dependent variables in the presence of outliers and possible departures from the normality assumption of the error distribution in robust regression analysis. We compared robust and classical variable selection. Here, as a classics selection criteria we used Cp, AICC and AICF which we proposed. Besides we used Andrews, Huber and Hampel M-estimators in computing of the robust variable selection criteria.

On a fast, robust estimator of the mode: Comparisons to other robust estimators with applications

Advances in computing power enable more widespread use of the mode, which is a natural measure of central tendency since it is not influenced by the tails in the distribution. The properties of the half-sample mode, which is a simple and fast estimator of the mode of a continuous distribution, are studied. The half-sample mode is less sensitive to outliers than most other estimators of location, including many other low-bias estimators of the mode. Its breakdown point is one half, equal to that of the median. However, because of its finite rejection point, the half-sample mode is much less sensitive to outliers that are all either greater or less than the other values of the sample. This is confirmed by applying the mode estimator and the median to samples drawn from normal, lognormal, and Pareto distributions contaminated by outliers. It is also shown that the half-sample mode, in combination with a robust scale estimator, is a highly robust starting point for iterative robust location estimators such as Huber's M-estimator. The half-sample mode can easily be generalized to modal intervals containing more or less than half of the sample. An application of such an estimator to the finding of collision points in high-energy proton–proton interactions is presented.

Minimax robust M-beamforming for radar array with antenna switching

We address the problem of estimating the movement parameters in non-Gaussian noise environments using an active radar array with antenna switching. The switch radar system is composed of a single transmitter, M receiving antennas, and a single receiving channel. We propose a new three-dimensional (3-D) power function which is exploited for beamforming and estimation of parameters of moving targets. Maximum peaks of this power function are used for estimation of the angle, velocity, and range of multiple moving targets. The robustness of the beamforming and the estimates is enabled by using Huber's M-estimates of the echoed transmitted radar signal. Two robust beamformers are studied: robust Huber and robust approximate median. Simulation demonstrates that the proposed robust beamformers provide a decreased sensitivity of the estimates with respect to noise components having heavy-tailed distributions.

Studentized bootstrap confidence intervals based on M-estimates

This article reviews and applies saddlepoint approximations to studentized confidence intervals based on robust M-estimates. The latter are known to be very accurate without needing standard theory assumptions. As examples, the classical studentized statistic, the studentized versions of Huber's M-estimate of location, of its initially MAD scaled version and of Huber's proposal 2 are considered. The aim is to know whether the studentized statistics yield robust confidence intervals with coverages close to nominal, with short intervals. The results of an extensive simulation study and the recommendations for practical use given in this article may fill gaps in the current literature and stimulate further discussion and research.

ONE-SAMPLE EXPLORATORY PROCEDURES AFTER SEARCHING THE UNDERLYING DISTRIBUTION

As statistical estimation procedures for location, the sample mean, Hodges and Lehman's R-estimator, and Huber's M-estimator are introduced in a one-sample model. The asymptotic distributional theory for the three estimators and simulated mean squared errors give the features of the respective estimators depending on the underlying distribution. Based on the features, we propose an estimation procedure selecting one of the three estimators after searching a distribution near to the underlying distribution. It is shown that the mean squared error of the new estimator is more stable than the three estimators.Next, as distribution-free test procedures, the conditional t-test, Wilcoxon's signed rank test, and the M-test are introduced. Asymptotic relative efficiency and simulated power of the respective tests are investigated. Based on their features, we propose a stable test procedure selecting one of the three tests after searching a distribution near to the underlying distribution.

Power system Huber M-estimation with equality and inequality constraints

The solution to the power system state estimation problem using the Huber M-estimator has been previously discussed. In earlier methods, the state estimation problem has been formulated as an unconstrained nonlinear program. The power systems literature reports solution to this problem using an iteratively re-weighted least squares technique. In this paper, the Huber M-estimator is formulated as a constrained nonlinear programming problem in which all functions are twice continuously differentiable. Such a formulation entails two advantages. Firstly, it enables accurate modelling of both zero bus injections and realistic system limits via equality and inequality constraints. Secondly, the differentiability property allows the system state to be conveniently computed via a primal-dual interior-point approach. Simulations on standard power systems show that even in the presence of bad data, the equality constraints in the Huber M-estimator effectively model the zero bus injections. Moreover, the numerical results reveal that the enforcement of practical system limits via inequality constraints can be useful in the absence of complete system observability.

Computation of Huber's M-estimates for a block-angular regression problem

Huber's M-estimation technique is applied to a block-angular regression problem, which may arise from some applications. A recursive, modified Newton approach to computing the estimates is presented. The structure of the problem is exploited to make the algorithm efficient. It is shown how to efficiently compute a descent search direction by using updating/downdating techniques for matrix factorizations. Numerical test results suggest that the proposed approach is effective.

A Recursive Least M-Estimate Algorithm for Robust Adaptive Filtering in Impulsive Noise: Fast Algorithm and Convergence Performance Analysis

This paper studies the problem of robust adaptive filtering in impulsive noise environment using a recursive least M-estimate algorithm (RLM). The RLM algorithm minimizes a robust M-estimator-based cost function instead of the conventional mean square error function (MSE). Previous work has showed that the RLM algorithm offers improved robustness to impulses over conventional recursive least squares (RLS) algorithm. In this paper, the mean and mean square convergence behaviors of the RLM algorithm under the contaminated Gaussian impulsive noise model is analyzed. A lattice structure-based fast RLM algorithm, called the Huber Prior Error Feedback-Least Squares Lattice (H-PEF-LSL) algorithm is derived. Part of the H-PEF-LSL algorithm was presented in ICASSP 2001. It has an order O(N) arithmetic complexity, where N is the length of the adaptive filter, and can be viewed as a fast implementation of the RLM algorithm based on the modified Huber M-estimate function and the conventional PEF-LSL adaptive filtering algorithm. Simulation results show that the transversal RLM and the H-PEF-LSL algorithms have better performance than the conventional RLS and other RLS-like robust adaptive algorithms tested when the desired and input signals are corrupted by impulsive noise. Furthermore, the theoretical and simulation results on the convergence behaviors agree very well with each other.

Finite Computation of the ℓ1 Estimator from Huber’s M-Estimator in Linear Regression

We review and extend previous work on the approximation of the linear l1 estimator by the Huber M-estimator based on the algorithms proposed by Clark and Osborne [7], and Madsen and Nielsen [12]. Although the Madsen-Nielsen algorithm is a promising one, it is guaranteed to terminate finitely under certain assumptions. We describe a variant of the Madsen-Nielsen algorithm to compute the l1 estimator from the Huber M-estimator in a finite number of steps without any restrictive steps nor assumptions. Summary computational results are given.

Robust Parameter Estimation for Uncertain Gross-Error Models

Many assumptions commonly made in science and engineering problems are at most approximations to reality and they do not always hold unfortunately. Recognizing this fact, the concept of robustness has been attracted and robust procedures have been developed to cover this fact. One of the most important ideas on this direction is Huber's M-estimator (maximum likelihood type estimator). Identifying the neighbourhoods of stochastic models in terms of the class of ε-contaminated probability distribution, he derived an estimator that minimizes the maximum degradation of performance possible for an ॉ-deviation from the assumption. This idea, however, is not applicable in practice since the exact value of the gross error ε is not known. In this paper, an M-estimator applicable in such situations is derived. A numerical example is presented for illustrating the proposed idea.

Uniform strong consistency of robust estimators

In the robustness framework, the distribution underlying the data is not totally specified and, therefore, it is convenient to use estimators whose properties hold uniformly over the whole set of possible distributions. In this paper, we give two general results on uniform strong consistency and apply them to study the uniform consistency of some classes of robust estimators over contamination neighborhoods. Some instances covered by our results are Huber's M-estimators, quantiles, or generalized S-estimators.

Adaptive robust detection for CDMA using a genetic algorithm

A novel adaptive robust multi-user detector for code-division multiple access (CDMA) using a real-coded genetic algorithm (GA) is proposed. The GA implicitly implements Huber's M-estimator and is robust against heavy-tailed impulsive noise. The novel feature of this GA-based detector lies in joint symbol detection and adaptive estimation of the cut-off parameter of the M-estimator's object function through a GA optimisation strategy. In particular, the GA-based detector carries out a multipoint search by manipulating and maintaining a population of candidate solutions for different values of the cut-off parameter to encourage information formation and exchange. Since the GA treats the cut-off parameter as one of its optimisation parameters, the need for a separate channel estimator is thus eliminated. Simulation results are provided to examine the evolutionary behaviour and the detection performance of the proposed GA-based detector. It is shown that the GA approach provides good adaptive impulsive noise rejection capability.

Robust CDMA multiuser detection using a neural-network approach

Abstract-Recently, a robust version of the linear decorrelating detector (LDD) based on the Huber's M-estimation technique has been proposed. In this paper, we first demonstrate the use of a three-layer recurrent neural network (RNN) to implement the LDD without requiring matrix inversion. The key idea is based on minimizing an appropriate computational energy function iteratively. Second, it will be shown that the M-decorrelating detector (MDD) can be implemented by simply incorporating sigmoidal neurons in the first layer of the RNN. A proof of the redundancy of the matrix inversion process is provided and the computational saving in realistic network is highlighted. Third, we illustrate how further performance gain could be achieved for the subspace-based blind MDD by using robust estimates of the signal subspace components in the initial stage. The impulsive noise is modeled using non-Gaussian alpha-stable distributions, which do not include a Gaussian component but facilitate the use of the recently proposed geometric signal-to-noise ratio (G-SNR). The characteristics and performance of the proposed neural-network detectors are investigated by computer simulation.

A measure of relative efficiency for location of a single sample

The question of how much to trim or which weighting constant to use are practical considerations in applying robust methods such as trimmed means (L-estimators) and Huber statistics (M-estimators). An index oflocation relative efficiency (LRE), which is a ratio of the narrowness of resulting confidence intervals, was applied to various trimmed means and Huber M-estimators calculated on seven representative data sets from applied education and psychology research. On the basis of LREs, lightly trimmed means were found to be more efficient than heavily trimmed means, but Huber M-estimators systematically produced narrower confidence intervals. The weighting constant of ψ = 1.28 was found to be superior to various competitors suggested in the literature for n < 50.

ROBUST ESTIMATION OF THE MEDIAN LETHAL DOSE

Alternatives to M-estimation for robust estimation of the median lethal dose in biological assays are developed. A class of link functions based on the Student-tdistribution is proposed, where degrees of freedom are estimated from the data by maximum likelihood. Other alternatives include slash and finite mixture distributions. For bioassays from a pharmaceutical company, these methods extend the standard probit and logistic models, as well as the Huber's M-estimator. They are also applied to several standard examples from the literature.

A simple duality proof in convex quadratic programming with a quadratic constraint, and some applications

In this paper a simple derivation of duality is presented for convex quadratic programs with a convex quadratic constraint. This problem arises in a number of applications including trust region subproblems of nonlinear programming, regularized solution of ill-posed least squares problems, and ridge regression problems in statistical analysis. In general, the dual problem is a concave maximization problem with a linear equality constraint. We apply the duality result to: (1) the trust region subproblem, (2) the smoothing of empirical functions, and (3) to piecewise quadratic trust region subproblems arising in nonlinear robust Huber M-estimation problems in statistics. The results are obtained from a straightforward application of Lagrange duality.

Censored regressión models with double exponential error distributions: as iterative estimation procedure based on medians for correcting bias

In this paper, we consider a simple iterative estimation procedure for censored regression models with symmetrical exponential error distributions. Although each step requires to impute the censored data with conditional medians, its tractability is guaranteed as well as its convergence at geometrical rate. Finally, as the final estimate coincides with a Huber M-estimator, its consistency and asymptotic normality are easily proved.

Algorithms for box constrained M-estimation: Fitting large data sets with application to geodynamics

Because of occurrence of ill-conditioning and outliers, use of direct Least Squares fit is now in decline, while robust M-estimators are currently attracting attention. We present here new algorithms based on the Spingarn Partial Inverse proximal decomposition method for L1 and Huber-M estimation that take into account both primal and dual aspects of the underlying optimization problem. The result is a family of highly parallel algorithms. Globally convergent, they are attractive for large scale problems as encountered in geodesy, especially in the field of Earth Orientation data analysis. The method is extended to handle box constrained problems. To obtain an efficient implementation, remedies are introduced to ensure efficiency in the case of models with less than full rank. Numerical results are discussed. Robust data pre-conditioning is shown to induce faster algorithm convergence. Practical implementation aspects are presented with application to series describing the Earth Rotation.

Robust linear and support vector regression

The robust Huber M-estimator, a differentiable cost function that is quadratic for small errors and linear otherwise, is modeled exactly, in the original primal space of the problem, by an easily solvable simple convex quadratic program for both linear and nonlinear support vector estimators. Previous models were significantly more complex or formulated in the dual space and most involved specialized numerical algorithms for solving the robust Huber linear estimator. Numerical test comparisons with these algorithms indicate the computational effectiveness of the new quadratic programming model for both linear and nonlinear support vector problems. Results are shown on problems with as many as 20000 data points, with considerably faster running times on larger problems.

Fitting Data with Errors in All Variables Using the Huber M-estimator

This article is concerned with the problem of data fitting where the model is nonlinear in the free parameters, using the Huber M-estimator. Under the assumption that there are significant errors in all the variables, an efficient algorithm is developed. Some numerical examples are given.