Least Trimmed Squares Research Articles

A Genetic Algorithm Based Modification on the LTS Algorithm for Large Data Sets

The authors introduce an algorithm for estimating the least trimmed squares (LTS) parameters in large data sets. The algorithm performs a genetic algorithm search to form a basic subset that is unlikely to contain outliers. Rousseeuw and van Driessen (2006) suggested drawing independent basic subsets and iterating C-steps many times to minimize LTS criterion. The authors 'algorithm constructs a genetic algorithm to form a basic subset and iterates C-steps to calculate the cost value of the LTS criterion. Genetic algorithms are successful methods for optimizing nonlinear objective functions but they are slower in many cases. The genetic algorithm configuration in the algorithm can be kept simple because a small number of observations are searched from the data. An R package is prepared to perform Monte Carlo simulations on the algorithm. Simulation results show that the performance of the algorithm is suitable for even large data sets because a small number of trials is always performed.

An inversion approach for the inverse heat conduction problems

The inverse heat conduction problems (IHCP) analysis method provides an efficient approach for estimating the thermophysical properties of materials, the boundary conditions, or the initial conditions. Successful applications of the IHCP method depend mainly on the efficiency of the inversion algorithms. In this paper, a generalized objective functional, which has been developed using a generalized stabilizing functional and a combinational estimation that integrates the advantages of the least trimmed squares (LTS) estimation and the M-estimation, is proposed. The objective functional unifies the regularized M-estimation, the regularized least squares (LS) estimation, the regularized LTS estimation, the regularized combinational estimation of the LTS estimation and the M-estimation, and the regularized combinational estimation of the LS estimation and the M-estimation into a concise formula. The filled function method, which is coupled with the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm, is developed for searching a possible global optimal solution. Numerical simulations are implemented to evaluate the feasibility and effectiveness of the proposed algorithm. Favorable numerical performances and satisfactory results are observed, which indicates that the proposed algorithm is successful in solving the IHCP.

Robust Regression Estimates in the Prediction of Latent Variables in Structural Equation Models

The incorporation of the robust regression methods Least Median Square (LMS) and Least Trimmed Squares (LTS) is proposed in structural equation modeling. Results show that, in situations of high deviations of symmetry, the evaluated methods would be recommended for applications including smaller sample sizes.

Outlier detection under multicollinearity

Multicollinearity is a problem with dependence of regressors. In this paper, we investigate the functionality in identifying outliers of several robust methods based on residual weighting (least median of squares (LMS), least trimmed squares (LTS) and least weighted squares (LWS)) in the situation where the majority of the data suffer from multicollinearity. This is closely knit with the possibility of using LMS, LTS and LWS as tools for multicollinearity detection. It is shown that these methods fail in outlier detection (although they are ‘built’ for this purpose) with the increasing rate of multicollinearity. Theoretical results are also supported by a simulation study. Finally, we propose an alternative method, ridge LWS, that can be profitably used instead of LWS in the presence of multicollinearity.

Comparison of Two Robust Alternatives to the Box–Draper Determinant Criterion in Multiresponse Kinetic Parameter Estimation

Generalized least-squares and maximum likelihood approaches for parameter estimation in multivariate response models have been prevalent in the chemical kinetics literature to date. In contrast, robust alternatives have received considerably less attention. These methods safeguard against possible deviations from the assumptions, such as the presence of outliers or non-normality of the random errors. We compare, through Monte Carlo simulation, the performance of the classical Box–Draper determinant criterion (ML) to those of two robust estimators: the multivariate Huber’s M-estimator and the multivariate least-trimmed squares estimator (MLTS). Although the results are not entirely conclusive, overall, we find no compelling evidence for preferring any one of the two robust methods over the conventional ML estimates. At the same time, it was unexpected to find that ML is still reasonable under mild outlier contamination and mild deviations from normality. This notwithstanding, one loses nothing by comparing ML together with MLTS to cross-check each other as a safety measure.

An Estimation of Solar Radiation using Robust Linear Regression Method

The air temperature data is the most important component to estimate the solar radiation in photovoltaic systems. From the Malaysia Meteorological Department, the data of air temperature and solar radiation can be found the hourly, daily, monthly and also the annually. Based on solar radiation data for the past 26 years, the average monthly solar radiation was 5009.56 Wh/m2. It was greater than the normal solar radiation (3 kWh/m2), which indicates that the sky in Perlis was clear and very high solar radiation intensity for the months in the past 26 years. This paper presents an investigation of a relationship between solar radiation and temperature in Perlis, Northern Malaysia for the year of 2006. The Least Trimmed Squares (LTS) robust regression model was selected to estimate the solar radiation since the robust method is do not breakdown easily and are not much influenced by outliers.

Sparse Least Trimmed Squares Regression

Sparse model estimation is a topic of high importance in modern data analysis due to the increasing availability of data sets with a large number of variables. Another common problem in applied statistics is the presence of outliers in the data. This paper combines robust regression and sparse model estimation. A robust and sparse estimator is introduced by adding an L1 penalty on the coefficient estimates to the well known least trimmed squares (LTS) estimator. The breakdown point of this sparse LTS estimator is derived, and a fast algorithm for its computation is proposed. Both the simulation study and the real data example show that the LTS has better pre- diction performance than its competitors in the presence of leverage points.

A unified approach to autofocus and alignment for pattern localization using hybrid weighted Hausdorff distance

Pattern localization is a fundamental task in machine vision, and autofocus is a requirement for any automated inspection system by allowing greater variation in the distance from the camera to the object being imaged. In this paper, we propose a unified approach to simultaneous autofocus and alignment for pattern localization by extending the idea of image reference approach. Under the least trimmed squares (LTS) scheme, the proposed hybrid weighted Hausdorff distance (HWHD) is a robust similarity metric that combines the Hausdorff distance (HD) with the edge-amplitude normalized gradient (EANG) matching. The EANG is designed to characterize the different degrees of blur at the edge points for focus cues, immune to illumination variations between the reference and the target image. We experimentally illustrate its performance on simulated as well as real data.

Generalised multi-scale image reconstruction algorithm for electrical capacitance tomography

Successful applications of electrical capacitance tomography (ECT) depend mainly on the precision and speed of the image reconstruction algorithms. In this paper, based on the wavelet multi-scale analysis method, a generalised multi-scale model of considering the inaccuracy of the capacitance data and reconstruction model is proposed, in which the original inverse problem is decomposed into a sequence of inverse problems based on the scale variables and then solved successively from the largest scale to the smallest scale until the solution of the original inverse problem is found. A generalised multi-scale objective functional, which has been developed using the least trimmed squares (LTS) estimation and the M-estimation, is proposed. This objective functional unifies the regularised LTS estimation, the regularised M-estimations, the regularised least squares (LS) estimation, the regularised combinational estimation of the LTS estimation and the M-estimations, the regularised combinational estimation of the LS estimation and the M-estimations into a concise formulation. An efficient solver, which integrates the beneficial advantages of the homotopy algorithm, the harmony search (HS) algorithm that has been developed using the multi-harmony techniques based on the cooperation of solutions, and the particle collision (PC) algorithm, is designed for searching a possible global optimal solution. The proposed algorithm is tested by six typical reconstruction objects using a 12-electrode square sensor. Numerical results show the efficiency and superiority of the proposed algorithm in solving ECT image reconstruction problem. In the cases considered in this paper, good results that show great improvement in the spatial resolution and accuracy are observed. The spatial resolution of the reconstructed images by the proposed algorithm is enhanced and the artefacts in the reconstructed images can be eliminated effectively. Meanwhile, the reconstructed results derived from the noise-contaminated capacitance data indicate that the proposed algorithm is successful in dealing with the inaccurate property in the capacitance data.

Robust Nonparametric Regression via Sparsity Control With Application to Load Curve Data Cleansing

Nonparametric methods are widely applicable to statistical inference problems, since they rely on a few modeling assumptions. In this context, the fresh look advocated here permeates benefits from variable selection and compressive sampling, to robustify nonparametric regression against outliers - that is, data markedly deviating from the postulated models. A variational counterpart to least-trimmed squares regression is shown closely related to an L0-(pseudo)norm-regularized estimator, that encourages sparsity in a vector explicitly modeling the outliers. This connection suggests efficient solvers based on convex relaxation, which lead naturally to a variational M-type estimator equivalent to the least-absolute shrinkage and selection operator (Lasso). Outliers are identified by judiciously tuning regularization parameters, which amounts to controlling the sparsity of the outlier vector along the whole robustification path of Lasso solutions. Reduced bias and enhanced generalization capability are attractive features of an improved estimator obtained after replacing the L0-(pseudo)norm with a nonconvex surrogate. The novel robust spline-based smoother is adopted to cleanse load curve data, a key task aiding operational decisions in the envisioned smart grid system. Computer simulations and tests on real load curve data corroborate the effectiveness of the novel sparsity-controlling robust estimators.

Data Truncation Bias, Loss Firms, and Accounting Anomalies

ABSTRACTEx post trimming of extreme returns observations that are not data errors causes spurious inferences in tests of market efficiency and behavioral explanations for anomalies. Trimming causes a downward truncation bias in estimated mean returns that is stronger in ex ante subsamples with more loss firms and in which return distributions are more right-skewed. There is an asymmetric U-shaped relation between return right-skewness and loss frequency across deciles of negative return predictors (Accruals, ΔNOA, and NOA), and a downward sloping relationship for positive return predictors (CFO and FCF). Consequently, a least-trimmed square (LTS) 1 percent deletion of returns induces a spurious inverted-U-shaped relation between returns and negative predictors, and an exaggerated positive relation for positive predictors. Thus, the resulting trimmed relations do not reject behavioral explanations for these anomalies. Trimming also induces a spurious loss anomaly. These findings highlight that in return prediction studies, observations should not be deleted based upon the values of the dependent variable, only based upon clearly identified data errors.

Modeling of Nonlinear Aggregation for Information Fusion Systems with Outliers Based on the Choquet Integral

Modern information fusion systems essentially associate decision-making processes with multi-sensor systems. Precise decision-making processes depend upon aggregating useful information extracted from large numbers of messages or large datasets; meanwhile, the distributed multi-sensor systems which employ several geographically separated local sensors are required to provide sufficient messages or data with similar and/or dissimilar characteristics. These kinds of information fusion techniques have been widely investigated and used for implementing several information retrieval systems. However, the results obtained from the information fusion systems vary in different situations and performing intelligent aggregation and fusion of information from a distributed multi-source, multi-sensor network is essentially an optimization problem. A flexible and versatile framework which is able to solve complex global optimization problems is a valuable alternative to traditional information fusion. Furthermore, because of the highly dynamic and volatile nature of the information flow, a swift soft computing technique is imperative to satisfy the demands and challenges. In this paper, a nonlinear aggregation based on the Choquet integral (NACI) model is considered for information fusion systems that include outliers under inherent interaction among feature attributes. The estimation of interaction coefficients for the proposed model is also performed via a modified algorithm based on particle swarm optimization with quantum-behavior (QPSO) and the high breakdown value estimator, least trimmed squares (LTS). From simulation results, the proposed MQPSO algorithm with LTS (named LTS-MQPSO) readily corrects the deviations caused by outliers and swiftly achieves convergence in estimating the parameters of the proposed NACI model for the information fusion systems with outliers.

Robust estimation of parameter for fractal inverse problem

In this paper, some similarity measures for fractal image compression (FIC) are introduced, which are robust against noises. In the proposed methods, robust estimation technique from statistics is embedded into the encoding procedure of the fractal inverse problem to find the parameters. When the original image is corrupted by noises, we hope that the proposed scheme is insensitive to those noises presented in the corrupted image. This leads to a new concept of robust estimation of fractal inverse problem. The proposed least absolute derivation (LAD), least trimmed squares (LTS), and Wilcoxon FIC are the first attempt toward the design of robust fractal image compression which can remove the noises in the encoding process. The main disadvantage of the robust FIC is the computational cost. To overcome this drawback, particle swarm optimization (PSO) technique is utilized to reduce the searching time. Simulation results show that the proposed FIC is robust against the outliers in the image. Also, the PSO method can effectively reduce the encoding time while retaining the quality of the retrieved image.

Asymptotics of Bayesian median loss estimation

We establish the consistency, asymptotic normality, and efficiency for estimators derived by minimizing the median of a loss function in a Bayesian context. We contrast this procedure with the behavior of two Frequentist procedures, the least median of squares (LMS) and the least trimmed squares (LTS) estimators, in regression problems. The LMS estimator is the Frequentist version of our estimator, and the LTS estimator approaches a median-based estimator as the trimming approaches 50% on each side. We argue that the Bayesian median-based method is a good tradeoff between the two Frequentist estimators.

Matrix strategies for computing the least trimmed squares estimation of the general linear and SUR models

An algorithm for computing the exact least trimmed squares (LTS) estimator of the standard regression model has recently been proposed. The LTS algorithm is adapted to the general linear and seemingly unrelated regressions models with possible singular dispersion matrices. It searches through a regression tree to find the optimal estimates and has combinatorial complexity. The model is formulated as a generalized linear least squares problem. Efficient matrix techniques are employed to update the generalized residual sum of squares of a subset model. Specifically, the new algorithm utilizes previous computations to update a generalized QR decomposition by a single row. The sparse structure of the model is exploited. Theoretical measures of computational complexity are provided. Experimental results confirm the ability of the new algorithms to identify outlying observations.

An Exact Least Trimmed Squares Algorithm for a Range of Coverage Values

A new algorithm to solve exact least trimmed squares (LTS) regression is presented. The adding row algorithm (ARA) extends existing methods that compute the LTS estimator for a given coverage. It employs a tree-based strategy to compute a set of LTS regressors for a range of coverage values. Thus, prior knowledge of the optimal coverage is not required. New nodes in the regression tree are generated by updating the QR decomposition of the data matrix after adding one observation to the regression model. The ARA is enhanced by employing a branch and bound strategy. The branch and bound algorithm is an exhaustive algorithm that uses a cutting test to prune nonoptimal subtrees. It significantly improves over the ARA in computational performance. Observation preordering throughout the traversal of the regression tree is investigated. A computationally efficient and numerically stable calculation of the bounds using Givens rotations is designed around the QR decomposition, avoiding the need to explicitly update the triangular factor when an observation is added. This reduces the overall computational load of the preordering device by approximately half. A solution is proposed to allow preordering when the model is underdetermined. It employs pseudo-orthogonal rotations to downdate the QR decomposition. The strategies are illustrated by example. Experimental results confirm the computational efficiency of the proposed algorithms. Supplemental materials (R package and formal proofs) are available online.

Adaptive trimmed likelihood estimation in regression

Summary In this paper we derive an asymptotic normality result for an adaptive trimmed likelihood estimator of regression starting from initial high breakdownpoint robust regression estimates. The approach leads to quickly and easily computed robust and efficient estimates for regression. A highlight of the method is that it tends automatically in one algorithm to expose the outliers and give least squares estimates with the outliers removed. The idea is to begin with a rapidly computed consistent robust estimator such as the least median of squares (LMS) or least trimmed squares (LTS) or for example the more recent MM estimators of Yohai. Such estimators are now standard in statistics computing packages, for example as in SPLUS or R. In addition to the asymptotics we provide data analyses supporting the new adaptive approach. This approach appears to work well on a number of data sets and is quicker than the related brute force adaptive regression approach described in Clarke (2000). This current approach builds on the work of Bednarski and Clarke (2002) which considered the asymptotics for the location estimator only.

Robust Calibration for Localization in Clustered Wireless Sensor Networks

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> This paper presents a robust calibration procedure for clustered wireless sensor networks. Accurate calibration of between-node distances is one crucial step in localizing sensor nodes in an ad-hoc sensor network. The calibration problem is formulated as a parameter estimation problem using a linear calibration model. For reducing or eliminating the unwanted influence of measurement corruptions or outliers on parameter estimation, which may be caused by sensor or communication failures, a robust regression estimator such as the least-trimmed squares (LTS) estimator is a natural choice. Despite the availability of the FAST-LTS routine in several statistical packages (e.g., R, S-PLUS, SAS), applying it to the sensor network calibration is not a simple task. To use the FAST-LTS, one needs to input a trimming parameter, which is a function of the sensor redundancy in a network. Computing the redundancy degree and subsequently solving the LTS estimation both turn out to be computationally demanding. Our research aims at utilizing some cluster structure in a network configuration in order to do robust estimation more efficiently. We present two algorithms that compute the exact value and a lower bound of the redundancy degree, respectively, and an algorithm that computes the LTS estimation. Two examples are presented to illustrate how the proposed methods help alleviate the computational demands associated with robust estimation and thus facilitate robust calibration in a sensor network. </para>

A Robust Method for Relative Gravity Data Estimation with High Efficiency

When gravimetric data observations have outliers, using standard least squares (LS) estimation will likely give poor accuracies and unreliable parameter estimates. One of the typical approaches to overcome this problem consists of using the robust estimation techniques. In this paper, we modified the robust estimator of Gervini and Yohai (2002) called REWLSE (Robust and Efficient Weighted Least Squares Estimator), which combines simultaneously high statistical efficiency and high breakdown point by replacing the weight function by a new weight function. This method allows reducing the outlier impacts and makes more use of the information provided by the data. In order to adapt this technique to the relative gravity data, weights are computed using the empirical distribution of the residuals obtained initially by the LTS (Least Trimmed Squares) estimator and by minimizing the mean distances relatively to the LS-estimator without outliers. The robustness of the initial estimator is maintained by adapted cut-off values as suggested by the REWLSE method which allows also a reasonable statistical efficiency. Hereafter we give the advantage and the pertinence of REWLSE procedure on real and semi-simulated gravity data by comparing it with conventional LS and other robust approaches like M- and MM-estimators.

Outlier detection by means of robust regression estimators for use in engineering science

This study compares the ability of different robust regression estimators to detect and classify outliers. Well-known estimators with high breakdown points were compared using simulated data. Mean success rates (MSR) were computed and used as comparison criteria. The results showed that the least median of squares (LMS) and least trimmed squares (LTS) were the most successful methods for data that included leverage points, masking and swamping effects or critical and concentrated outliers. We recommend using LMS and LTS as diagnostic tools to classify outliers, because they remain robust even when applied to models that are heavily contaminated or that have a complicated structure of outliers.