Least Trimmed Squares Estimator Research Articles

A model where the least trimmed squares estimator is maximum likelihood

Abstract The least trimmed squares (LTS) estimator is a popular robust regression estimator. It finds a subsample of h ‘good’ observations among n observations and applies least squares on that subsample. We formulate a model in which this estimator is maximum likelihood. The model has ‘outliers’ of a new type, where the outlying observations are drawn from a distribution with values outside the realized range of h ‘good’, normal observations. The LTS estimator is found to be h1/2 consistent and asymptotically standard normal in the location-scale case. Consistent estimation of h is discussed. The model differs from the commonly used ϵ-contamination models and opens the door for statistical discussion on contamination schemes, new methodological developments on tests for contamination as well as inferences based on the estimated good data.

Robust restricted Liu estimator in censored semiparametric linear models

It is not unusual to have outliers and multicollinearity simultaneously in censored semiparametric linear models. In this paper for dealing with multicollinearity and outliers we introduce a family of robust censored Liu and non-Liu type of estimates for the regression parameter when some non-stochastic linear restrictions are imposed. The proposed robust estimators is based on least trimmed squares (LTS) method. The efficiency of LTS estimator statistically is more than the well-known least median squares (LMS) estimator. Unfortunately the computational complexity of LTS is less well understood than that of LMS in which the objective is to minimize the median squared residual. Here, we provide the robust estimators for linear and non-linear parts of the censored model based on robust shrinkage Liu estimators. The performance of proposed estimators compared to ordinary estimators numerically is evaluated by Monte Carlo simulation studies. We further illustrate the our procedures by an application.

On the least trimmed squares estimators for JS circular regression model

The least trimmed squares (LTS) estimation has been successfully used in the robust linear regression models. This article extends the LTS estimation to the Jammalamadaka and Sarma (JS) circular regression model. The robustness of the proposed estimator is studied and the used algorithm for computation is discussed. Simulation studied, and real data show that the proposed robust circular estimator effectively fits JS circular models in the presence of vertical outliers and leverage points.

Power Systems Decomposition for Robustifying State Estimation Under Cyber Attacks

This paper proposes an automatic algorithm for the on-line implementation of a robust static state estimator on large power systems. The proposed algorithm maximizes the number of outliers and cyber-attacks that the estimator can resist while giving reliable estimates. The large power system is decomposed in several islands or subsystems, and a highly robust regression estimator, namely the least trimmed squares estimator (LTS), is implemented on each island to detect bad data. Executing the estimators in parallel will greatly reduce the computation time of the robust static state estimator. The introduced method is compared with two cycle detection graph-theory approaches, namely depth-first search (DFS) and minimum spanning tree (MST), which have been adapted here for power state estimation. Simulations on IEEE 14, 30, 57, 118, 145, and 300 bus systems show the superior performance of the proposed algorithm over the adapted DFS and MST. The algorithm could reduce significantly the number and size of cycles in the system. Furthermore, the number of detected outliers, and attacks is maximized while the observability of the system is ensured. Attacks or outliers on both measurements, and topology of the grid are detected as well.

Robust Method in Multiple Linear Regression Model on Diabetes Patients

This paper is focusing on the application of robust method in multiple linear regression (MLR) model towards diabetes data. The objectives of this study are to identify the significant variables that affect diabetes by using MLR model and using MLR model with robust method, and to measure the performance of MLR model with/without robust method. Robust method is used in order to overcome the outlier problem of the data. There are three robust methods used in this study which are least quartile difference (LQD), median absolute deviation (MAD) and least-trimmed squares (LTS) estimator. The result shows that multiple linear regression with application of LTS estimator is the best model since it has the lowest value of mean square error (MSE) and mean absolute error (MAE). In conclusion, plasma glucose concentration in an oral glucose tolerance test is positively affected by body mass index, diastolic blood pressure, triceps skin fold thickness, diabetes pedigree function, age and yes/no for diabetes according to WHO criteria while negatively affected by the number of pregnancies. This finding can be used as a guideline for medical doctors as an early prevention of stage 2 of diabetes.

A Comparative Study between Ridge MM and Ridge Least Trimmed Squares Estimators in Handling Multicollinearity and Outliers

Common problems found in multiple linear regression models are the existence of multicollinearity and outliers. These obstacles usually produce undesirable effects on least squares estimators. Ridge regression estimator is suggested in handling severe multicollinearity while robust estimators such as MM estimator and Least Trimmed Squares (LTS) estimator are recommended in tackling the outlier issues. An even worse scenario is when these two problems occur simultaneously. Combination of both leads to robust ridge regression methods which can be used to handle both conditions simultaneously. In this study, a comparative investigation is carried out to compare the performance between ridge MM and ridge LTS estimators. The Root Mean Square Error (RMSE) and Bias are obtained for each estimator to compare their performances. By using simulation study, Laplace and Cauchy distributions are used in introducing outliers to the simulated data with high multicollinearity ρ = 0.90, 0.95 and 0.98 for sample sizes n=25, 50 and 100. From the results, it is found that Ridge LTS is the best estimator for many combinations of error distributions and degrees of multicollinearity. Similar results were obtained when using two sets of real data.

Models Where the Least Trimmed Squares and Least Median of Squares Estimators Are Maximum Likelihood

The Least Trimmed Squares (LTS) and Least Median of Squares (LMS) estimators are popular robust regression estimators. The idea behind the estimators is to find, for a given h, a sub-sample of h 'good' observations among n observations and estimate the regression on that sub-sample. We find models, based on the normal or the uniform distribution respectively, in which these estimators are maximum likelihood. We provide an asymptotic theory for the location-scale case in those models. The LTS estimator is found to be h1/2 consistent and asymptotically standard normal. The LMS estimator is found to be h consistent and asymptotically Laplace.

Feasible robust estimator in restricted semiparametric regression models based on the LTS approach

ABSTRACTUnder some nonstochastic linear restrictions based on either additional information or prior knowledge in a semiparametric regression model, a family of feasible generalized robust estimators for the regression parameter is proposed. The least trimmed squares (LTS) method proposed by Rousseeuw as a highly robust regression estimator is a statistical technique for fitting a regression model based on the subset of h observations (out of n) whose least-square fit possesses the smallest sum of squared residuals. The coverage h may be set between n/2 and n. The LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. For practical purpose, it is assumed that the covariance matrix of the error term is unknown and thus feasible estimators are replaced. Then, we develop an algorithm for the LTS estimator based on feasible methods. Through the Monte Carlo simulation studies and a real data example, performance of the feasible type of robust estimators is compared with the classical ones in restricted semiparametric regression models.

Robust ridge estimator in restricted semiparametric regression models

In this paper, ridge and non-ridge type estimators and their robust forms are defined in the semiparametric regression model when the errors are dependent and some non-stochastic linear restrictions are imposed under a multicollinearity setting. In the context of ridge regression, the estimation of shrinkage parameter plays an important role in analyzing data. Another common problem in applied statistics is the presence of outliers in the data besides multicollinearity. In this respect, we propose some robust estimators for shrinkage parameter based on least trimmed squares (LTS) method. Given a set of n observations and the integer trimming parameter h≤n, the LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. The LTS estimator is closely related to the well-known least median squares (LMS) estimator in which the objective is to minimize the median squared residual. Although LTS estimator has the advantage of being statistically more efficient than LMS estimator, the computational complexity of LTS is less understood than LMS. Here, we extract the robust estimators for linear and nonlinear parts of the model based on robust shrinkage estimators. It is shown that these estimators perform better than ordinary ridge estimator. For our proposal, via a Monté-Carlo simulation and a real data example, performance of the ridge type of robust estimators are compared with the classical ones in restricted semiparametric regression models.

Ordinary Least Squares and Robust Estimators in Linear Regression: Impacts of Outliers, Error and Response Contaminations

The Ordinary Least Squares Estimator (OLSE) is the best method for linear regression if the classical assumptions are satis ed for estimating weights. When these assumptions are violated, the robust methods give more reliable estimates while the OLSE is strongly a ected adversely. In order to assess the sensitivity of some estimators using more than ve criteria, a secondary dataset on Anthropometric measurements from Komfo Anokye Teaching Hospital, Kumasi-Ghana, is used. In this study, we compare the performance of the Huber Maximum Likelihood Estimator (HMLE), Least Trimmed Squares Estimator (LTSE), S Estimator (SE) and Modi ed Maximum Likelihood Estimator (MMLE) relative to the OLSE when the dataset has normal errors; 10, 20 and 30 percent outliers; 20% error contamination and lognormal contamination in the response variable. In the assessment, we use coe cients and their standard errors, relative e ciencies, Root Mean Square Errors, and the coe cients of determination of the estimators. We also use the power of the test to assess the e ects of the aberrations on the post hoc power analysis of the estimators. The results show the SE and MMLE outperform the HMLE and LTSE while the OLSE breaks down completely. The LTSE performs well when the trimming is done to eliminate only the outliers. Also, SE and MMLE resist the e ect of all aberrations in the data and also have good post hoc power analysis.

Leverages, Outliers and the Performance of Robust Regression Estimators

This study compared the performance of some robust regression methods and the Ordinary Least Squares Estimator (OLSE). The estimators were compared using varied levels of leverages and vertical outliers in the predictors and the dependent variables. An anthropometric dataset on total body fat with height, Body Mass Index (BMI), Triceps Skin-fold(TS), and arm fat as percent composition of the body (parmfat), as the predictors. The e ects of outliers and leverages on the estimators, were investigated at (5% leverages and 10% vertical outliers, 5% leverages with 15% vertical outliers). The criteria for the comparison: coe cients, Root Mean Square Error (RMSE), Relative E ciencies (RE), coe cients of determination (R-squared) and power of the test. The ndings from this study revealed that, OLSE was a ected by both outliers and leverages whilst Huber Maximum likelihood Estimator (HME) was a ected by leverages. The Least Trimmed Squares Estimator (LTSE) was slightly a ected by high perturbations of outliers and leverages. The study also showed that Modi ed Maximum likelihood Estimator (MME) and S Estimator (SE) were robust to all levels of outliers and leverage perturbations. Therefore leverages and outliers in datasets do a ect the post hoc power analysis of the methods which cannot resist them.

SOCP relaxation bounds for the optimal subset selection problem applied to robust linear regression

This paper deals with the problem of finding the globally optimal subset of h elements from a larger set of n elements in d space dimensions so as to minimize a quadratic criterion, with an special emphasis on applications to computing the Least Trimmed Squares Estimator (LTSE) for robust regression. The computation of the LTSE is a challenging subset selection problem involving a nonlinear program with continuous and binary variables, linked in a highly nonlinear fashion. The selection of a globally optimal subset using the branch and bound (BB) algorithm is limited to problems in very low dimension, typically d ≤ 5, as the complexity of the problem increases exponentially with d. We introduce a bold pruning strategy in the BB algorithm that results in a significant reduction in computing time, at the price of a negligeable accuracy lost. The novelty of our algorithm is that the bounds at nodes of the BB tree come from pseudo-convexifications derived using a linearization technique with approximate bounds for the nonlinear terms. The approximate bounds are computed solving an auxiliary semidefinite optimization problem. We show through a computational study that our algorithm performs well in a wide set of the most difficult instances of the LTSE problem.

Extended least trimmed squares estimator in semiparametric regression models with correlated errors

Under a semiparametric regression model, a family of robust estimates for the regression parameter is proposed. The least trimmed squares (LTS) method is a statistical technique for fitting a regression model to a set of points. Given a set of n observations and the integer trimming parameter , the LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. The LTS estimator is closely related to the well-known least median squares (LMS) estimator in which the objective is to minimize the median squared residual. Although LTS estimator has the advantage of being statistically more efficient than LMS estimator, the computational complexity of LTS is less understood than LMS. Here, we develop an algorithm for the LTS estimator. Through a Monte Carlo approach, performance of the robust estimates is compared with the classical ones in semiparametric regression models.

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.

Credit Support for Export: Robust Evidence from the Czech Republic

AbstractThe topic of this paper is quite a novel one – it is one of few empirical academic papers dealing with export credit. Moreover, it is the first analysis of this kind which focuses on transition economies. The paper deals with export credit promotion in the Czech Republic. The development and structure of Czech trade and export support is presented first, followed by an econometric analysis of the gravity model of Czech Republic trade. A panel of 160 countries in 1996–2008 is analysed and two gravity models of exports for the Czech Republic are estimated, the static model by fixed effects (LSDV estimator) and the dynamic model by System GMM. Due to ambiguous conclusions we assume that the behaviour of our explanatory variables is not uniform and our data set behaves as a mixture of countries with heterogeneous behaviour. This means that traditional techniques of estimation which include all observations into one model do not give significant results. Thus, we use robust techniques of estimation that solve the problem of heterogeneous patterns in data sets. Out of several possibilities we use the Least Trimmed Squares estimator (LTS) with a leverage point. We show that guarantees are a significant factor that influences positively the volume of exports in the Czech Republic. Moreover, there exist more variables that affect the size of exports in the Czech Republic. Market forces described by GDP, distance, political risk or gross fix capital formation are significant in our econometric model. We find that higher GDP, shorter distance or lower political risk have a positive impact on Czech exports.

Robust ridge and robust Liu estimator for regression based on the LTS estimator

In the multiple linear regression analysis, the ridge regression estimator and the Liu estimator are often used to address multicollinearity. Besides multicollinearity, outliers are also a problem in the multiple linear regression analysis. We propose new biased estimators based on the least trimmed squares (LTS) ridge estimator and the LTS Liu estimator in the case of the presence of both outliers and multicollinearity. For this purpose, a simulation study is conducted in order to see the difference between the robust ridge estimator and the robust Liu estimator in terms of their effectiveness; the mean square error. In our simulations, the behavior of the new biased estimators is examined for types of outliers: X-space outlier, Y-space outlier, and X-and Y-space outlier. The results for a number of different illustrative cases are presented. This paper also provides the results for the robust ridge regression and robust Liu estimators based on a real-life data set combining the problem of multicollinearity and outliers.

Sparse least trimmed squares regression for analyzing high-dimensional large data sets

Sparse model estimation is a topic of high importance in modern data analysis due to the increasing availability of data sets with a large number of variables. Another common problem in applied statistics is the presence of outliers in the data. This paper combines robust regression and sparse model estimation. A robust and sparse estimator is introduced by adding an $L_1$ penalty on the coefficient estimates to the well-known least trimmed squares (LTS) estimator. The breakdown point of this sparse LTS estimator is derived, and a fast algorithm for its computation is proposed. In addition, the sparse LTS is applied to protein and gene expression data of the NCI-60 cancer cell panel. Both a simulation study and the real data application show that the sparse LTS has better prediction performance than its competitors in the presence of leverage points.

The Influence Function of Penalized Regression Estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore it can be used to compute the asymptotic variance and the mean squared error. In this paper we compute the influence function, the asymptotic variance and the mean squared error for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.

An inversion approach for the inverse heat conduction problems

The inverse heat conduction problems (IHCP) analysis method provides an efficient approach for estimating the thermophysical properties of materials, the boundary conditions, or the initial conditions. Successful applications of the IHCP method depend mainly on the efficiency of the inversion algorithms. In this paper, a generalized objective functional, which has been developed using a generalized stabilizing functional and a combinational estimation that integrates the advantages of the least trimmed squares (LTS) estimation and the M-estimation, is proposed. The objective functional unifies the regularized M-estimation, the regularized least squares (LS) estimation, the regularized LTS estimation, the regularized combinational estimation of the LTS estimation and the M-estimation, and the regularized combinational estimation of the LS estimation and the M-estimation into a concise formula. The filled function method, which is coupled with the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm, is developed for searching a possible global optimal solution. Numerical simulations are implemented to evaluate the feasibility and effectiveness of the proposed algorithm. Favorable numerical performances and satisfactory results are observed, which indicates that the proposed algorithm is successful in solving the IHCP.

회귀모형에서 이상치 검색을 이용한 로버스트 변수변환방법

Transforming response variable is a general tool to adapt data to a linear regression model. However, it is well known that response transformations in linear regression are very sensitive to one or a few outliers. Many methods have been suggested to develop transformations that will not be influenced by potential outliers. Recently Cheng (2005) suggested to using a trimmed likelihood estimator based on the idea of the least trimmed squares estimator(LTS). However, the method requires presetting the number of outliers and needs many computations. A new method is proposed, that can solve the problems addressed and improve the robustness of the estimates. The method uses a stepwise procedure, suggested by Hadi and Simonoff (1993), to detect outliers that determine response transformations.