Least Trimmed Squares Research Articles

Large Sample Behavior of the Least Trimmed Squares Estimator

The least trimmed squares (LTS) estimator is popular in location, regression, machine learning, and AI literature. Despite the empirical version of least trimmed squares (LTS) being repeatedly studied in the literature, the population version of the LTS has never been introduced and studied. The lack of the population version hinders the study of the large sample properties of the LTS utilizing the empirical process theory. Novel properties of the objective function in both empirical and population settings of the LTS and other properties, are established for the first time in this article. The primary properties of the objective function facilitate the establishment of other original results, including the influence function and Fisher consistency. The strong consistency is established with the help of a generalized Glivenko–Cantelli Theorem over a class of functions for the first time. Differentiability and stochastic equicontinuity promote the establishment of asymptotic normality with a concise and novel approach.

Robust Liu Estimator Used to Combat Some Challenges in Partially Linear Regression Model by Improving LTS Algorithm Using Semidefinite Programming

Outliers are a common problem in applied statistics, together with multicollinearity. In this paper, robust Liu estimators are introduced into a partially linear model to combat the presence of multicollinearity and outlier challenges when the error terms are not independent and some linear constraints are assumed to hold in the parameter space. The Liu estimator is used to address the multicollinearity, while robust methods are used to handle the outlier problem. In the literature on the Liu methodology, obtaining the best value for the biased parameter plays an important role in model prediction and is still an unsolved problem. In this regard, some robust estimators of the biased parameter are proposed based on the least trimmed squares (LTS) technique and its extensions using a semidefinite programming approach. Based on a set of observations with a sample size of n, and the integer trimming parameter h ≤ n, the LTS estimator computes the hyperplane that minimizes the sum of the lowest h squared residuals. Even though the LTS estimator is statistically more effective than the widely used least median squares (LMS) estimate, it is less complicated computationally than LMS. It is shown that the proposed robust extended Liu estimators perform better than classical estimators. As part of our proposal, using Monte Carlo simulation schemes and a real data example, the performance of robust Liu estimators is compared with that of classical ones in restricted partially linear models.

Comparative Study in Controlling Outliers and Multicollinearity Using Robust Performance Jackknife Ridge Regression Estimator Based on Generalized-M and Least Trimmed Square Estimator

Regression analysis is one of the statistical methods used to determine the causal relationship between one or more explanatory variables to the affected variable. The problem that often occurs in regression analysis is that there are multicollonity and outliers. To deal with such problems can be solved using ridge regression analysis and robust regression. Ridge regression can solve the problem of multicollinearas by assigning a constant k to the matrix Z′Z. But in this method the resulting bias value is still high, so to overcome this problem, the jackknife ridge regression method is used. Meanwhile, to overcome outliers in the data using robust regression methods which have several estimation methods, two of which are the Generalized-M (GM) estimator and the Least Trimmed Square (LTS) estimator. The aim of the study is to solve the problem of multicollinearity and outliers simultaneously using robust jackknife ridge regression method with GM estimators and LTS estimators. The results showed that the robust ridge jackknife regression method with LTS estimator can control multicollinearity and outliers simultaneously better based on MSE, AIC and BIC values compared to the robust ridge jackknife regression method with GM estimators. This is indicated by the value MSE = -6.60371, AIC = 75.823 and BIC = 81.642 on LTS estimators that are of lower value than GM estimators.

Perbandingan Analisis Komponen Utama Robust Minimum Covarian Determinant dengan Least Trimmed Square pada Data Produk Domestik Regional Bruto

Regression analysis is a method to examine the relationship between variables and determine their influence. However, the problem of multicollinearity often arises in linear regression analysis and can cause interpretation problems. To handle multicollinearity, Principal Component Analysis (PCA) is used. However, this method has a weakness when the data contains outliers. Therefore, it was developed into robust PCA using the Minimum Covariance Determinant (MCD) method and the Least Trimmed Square (LTS) estimation method. This study uses Gross Regional Domestic Product data in Indonesia in 2020, which has problems with multicollinearity and outliers. This data is modeled using two robust PCA methods, namely MCD and LTS. The robust PCA model with MCD has an adjusted value of 87.87% and an MSE value of 0.0700. However, in the robust PCA regression model with LTS, the adjusted value is 98.93% and the MSE value is 0.0325. Thus, the effective method in handling multicollinearity and outliers is the LTS method because it shows better results.

Weighted Least Squares Regression with the Best Robustness and High Computability

A novel regression method is introduced and studied. The procedure weights squared residuals based on their magnitude. Unlike the classic least squares which treats every squared residual as equally important, the new procedure exponentially down-weights squared residuals that lie far away from the cloud of all residuals and assigns a constant weight (one) to squared residuals that lie close to the center of the squared-residual cloud. The new procedure can keep a good balance between robustness and efficiency; it possesses the highest breakdown point robustness for any regression equivariant procedure, being much more robust than the classic least squares, yet much more efficient than the benchmark robust method, the least trimmed squares (LTS) of Rousseeuw. With a smooth weight function, the new procedure could be computed very fast by the first-order (first-derivative) method and the second-order (second-derivative) method. Assertions and other theoretical findings are verified in simulated and real data examples.

A fast robust best subset regression

In this paper, we present a novel approach called the Fast Robust Best Subset Regression (FRBSR) procedure, specifically designed to tackle outliers in both covariates and response variables. Through the incorporation of C-steps, the FRBSR procedure not only enhances the robustness of the estimation process, but also relaxes the stringent assumptions imposed by the Enhanced Support Detection and Root Finding (ESDAR, Huang et al. (2018, 2021)) algorithm regarding the distribution of covariates and response variables. More specifically, we propose additional techniques, namely reweighted least squares (REWLS) and one-step weighted least squares (OSWLS), which build upon the L0 regularized least trimmed squares (LTS) method and aim to improve estimation efficiency. Compared to the majorization-minimization (MM, Liu et al. (2021)) principle for solving non-convex loss L2E, our proposed OSWLS requires only one step to compute the L0 regularized weighted least squares, resulting in reduced estimation error and computational complexity. Extensive simulations and real data analysis demonstrate the superior performance of our FRBSR procedure in terms of estimation accuracy and prediction compared to the latest robust methods.

Florbetapir PET Uptake in White Matter in Aging and Alzheimer’s Disease

AbstractBackgroundAmyloid PET tracers, typically used to track cerebral amyloid changes in Alzheimer’s Disease (AD) patients, have been recently suggested to measure cerebral white matter (WM) integrity due to structural similarity between myelin and amyloid. We previously reported significant associations of amyloid neuroimaging marker, Pittsburgh Compound B (PiB) PET, with WM hyperintensities as quantified by MRI and FLAIR data, in AD patients (Su et al. 2018, HAI). The goal of this study was to explore the association between Florbetapir (FBP) PET and WM integrity.MethodOur study population consisted of 673 participants from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) sorted into the following groups of clinical severity: (1) Normal Controls (NC) participants who are amyloid negative and cognitively normal (CDR = 0) (A‐CDR‐); (2) Preclinical (Pre) who are cognitively normal (CDR = 0) and amyloid positive (A+CDR0); (3) Symptomatic AD who are both CDR and amyloid positive (A+CDR+); and (4) Suspected Other Dementia group consisting of cognitively impaired but amyloid negative participants. We analyzed baseline biomarkers as function of stage of AD and age, comparing groups using ANOVA and Scheffe’s Post Hoc Analysis, and linear regression, respectively. Associations between white matter and amyloid FBP uptake were quantified using Pearson correlation with Least Trimmed Square (LTS) regression. Lastly, rate of change of amyloid and white matter uptake of FBP PET were quantified to assess longitudinal biomarker changes using a one sample t‐test.ResultCross‐sectionally, baseline FBP uptake in white matter significantly declines with age (p = 0.027) and advancing disease stage with significant differences found between the Preclinical and Symptomatic AD group (p<2.2E‐06); and NC and Symptomatic AD group (p<0.003). Significant associations were also observed between baseline FBP uptake in white matter and amyloid (p<0.001).Longitudinally, rate of change of FBP uptake in white matter progressively declined with advancing disease stage while rate of change of FBP uptake in amyloid increased and plateaued in the Symptomatic group of patients.ConclusionAltogether, our results show declining FBP uptake by white matter, both cross‐sectionally and longitudinally, through the stages of AD and with age. Our results could have methodological implications on the quantification of amyloid burden using WM as a reference region.

Least Trimmed Squares for Regression Models with Stable Errors

Least Trimmed Squares (LTS) is a robust regression method with respect to outliers. It is based on performing Ordinary Least Squares (OLS) estimates on sub-datasets and determining the optimal solution corresponding to the minimum sum of squared residuals. Since the method of LTS is based on OLS, errors in regression models have finite variance. This work aims to generalize LTS for heavy tail data with infinite variance. When errors have infinite variance, it is impossible to benefit from OLS estimates. We use the property of variance existence of most ordered errors to find an initial robust OLS estimate. We polish the LTS method with a maximum likelihood estimator based on the density function of order statistics and determine the optimal solution for stable regression models. The proposed algorithm is implemented for linear regression models.

A model where the least trimmed squares estimator is maximum likelihood

Abstract The least trimmed squares (LTS) estimator is a popular robust regression estimator. It finds a subsample of h ‘good’ observations among n observations and applies least squares on that subsample. We formulate a model in which this estimator is maximum likelihood. The model has ‘outliers’ of a new type, where the outlying observations are drawn from a distribution with values outside the realized range of h ‘good’, normal observations. The LTS estimator is found to be h1/2 consistent and asymptotically standard normal in the location-scale case. Consistent estimation of h is discussed. The model differs from the commonly used ϵ-contamination models and opens the door for statistical discussion on contamination schemes, new methodological developments on tests for contamination as well as inferences based on the estimated good data.

Outlier-robust methods for forecasting realized covariance matrices

This paper proposes two new approaches to improve the estimation of the coefficients of the multivariate HAR (MHAR) model with the primary purpose of improving forecast performance. A robust estimator of the covariance matrix is adopted to replace the realized covariance matrix while estimating the MHAR model. The robustness to outliers of the new estimator makes the OLS estimation scheme for the MHAR model more reliable. In addition, a robust estimation scheme is developed for the MHAR model, which is based on the multivariate least-trimmed squares method. Both approaches provide significant improvements in forecasting performance based on both statistical loss and portfolio outcomes. The forecast performance of the multivariate HARQ model can also be improved with the proposed approaches, as evidenced by robustness checks.

New approaches for outlier detection: The least trimmed squares adjustment

Classical outlier tests based on the least-squares (LS) have significant disadvantages in some situations. The adjustment computation and classical outlier tests deteriorate when observations include outliers. The robust techniques that are not sensitive to outliers have been developed to detect the outliers. Several methods use robust techniques such as M-estimators, L1- norm, the least trimmed squares etc. The least trimmed squares (LTS) among them have a high-breakdown point. After the theoretical explanation, the adjustment computation has been carried out in this study based on the least squares (LS) and the least trimmed squares (LTS). A certain polynomial with arbitrary values has been used for applications. In this way, the performances of these techniques have been investigated.

On the study of efficiency of exponential‐type estimator of population mean using robust regression methods

AbstractThis study aimed at enhancing the efficiency of Zaman estimators using exponential transformation technique. A new class of estimators was obtained using the concept of Bahl and Tuteja. The bias and mean squared error (MSE) of the new class of suggested estimators was derived up to second degree approximation. The empirical study through simulations was conducted using Normal, exponential, gamma, chi‐square and beta distributions under robust regression methods (Huber‐M, Huber‐MM, LTS (least trimmed squares) and LMS (least median of squares)) and the results revealed that proposed estimators were more efficient.

Analisis Regresi Robust Estimasi Least Trimmed Square dan Estimasi Maximum Likelihood pada Pemodelan IPM di Pulau Sulawesi

Sulawesi Island is one of the islands in Indonesia with the second lowest HDI after Papua in 2011-2018, so it is necessary to study more deeply the factors that affect the HDI on Sulawesi Island. One of the methods in modeling is the robust regression method that can be used even though there are outliers in the data. This study aimed to determine the factors that affect HDI in Sulawesi Island using a robust regression model of Least Trimmed Square (LTS) estimation and Maximum Likelihood (M) estimation and to find out the best method for modeling HDI in Sulawesi Island. The robust regression of the LTS estimate has a reasonably high breakdown point value compared to other estimates, while the M-estimate has a high efficiency. The results showed that the factors that affect HDI on the island of Sulawesi are Life Expectancy (UHH), Expected Years of Schooling (HLS), Average Years of Schooling (RLS), and Real Expenditures per capita (PPP). The best model in HDI modeling in Sulawesi Island is a robust regression model with LTS estimation where the standard error is 0.1513, and the R-Square is 99.78%. A very high R-Square value indicates that the ability of the UHH, HLS, RLS, and PPP variables to explain HDI is very good.

A comparative analysis of the outliers influence using GMM estimation based on dynamic panel data model

ABSTRACT This study proposes a novel approach by combining the first differenced generalized method of moments method (DIF-GMM) to remove the estimates bias in the dynamic panel data model and the least trimmed squares (LTS) to control outlier influence. The combination of these two methods is referred to as DIF-GMM+LTS. We apply this approach to examine the influence of outliers on the effect of financial development on economic growth. Our results show a counter-intuitive evidence that the bank development negatively affects economic growth when the outlier influence is ignored. However, the bank development exhibits a positive influence on economic growth once the proposed approach DIF-GMM+LTS is adopted. Also, stock market development shows a positive effect on economic growth regardless of the outliers.

UV3D: Underwater Video Stream 3D Reconstruction Based on Efficient Global SFM

With the increasing demand for underwater resource exploration, three-dimensional (3D) reconstruction technology is important for searching for lost underwater civilizations, underwater shipwrecks, or seabed structures. However, faced with the limitations of underwater unmanned systems in terms of energy, bandwidth, and transmission delay, 3D reconstruction technology based on video streams as direct data will not work well. We propose a terminal image processing strategy to save data transmission time and cost and to obtain 3D scene information as soon as possible. Firstly, we propose an adaptive threshold key frame extraction algorithm based on clustering, which extracts key frames from the video stream as structure from motion (SFM) image sequences. On this basis, we enhance the underwater images with sufficient and insufficient illumination to improve the image quality and obtain a better visual effect in the 3D reconstruction step. Additionally, we choose global SFM to construct the scene and propose a faster rotation averaging method, the least trimmed square rotation averaging (LTS-RA) method, based on the least trimmed squares (LTS) and L1RA methods. It is proven to reduce 19.97% of the time through experiments. Finally, our experiments demonstrate that the dense point cloud saves about 70% of the transmission cost compared to video streaming.

Cross-Sectional Analysis of Impulse Indicator Saturation Method for Outlier Detection Estimated via Regularization Techniques with Application of COVID-19 Data.

Impulse indicator saturation is a popular method for outlier detection in time series modeling, which outperforms the least trimmed squares (LTS), M-estimator, and MM-estimator. However, using the IIS method for outlier detection in cross-sectional analysis has remained unexplored. In this paper, we probe the feasibility of the IIS method for cross-sectional data. Meanwhile, we are interested in forecasting performance and covariate selection in the presence of outliers. IIS method uses Autometrics techniques to estimate the covariates and outlier as the number of covariates P > n observations. Besides Autometrics, regularization techniques are a well-known method for covariate selection and forecasting in high-dimensional analysis. However, the efficiency of regularization techniques for the IIS method has remained unexplored. For this purpose, we explore the efficiency of regularization techniques for out-of-sample forecast in the presence of outliers with 6 and 4 standard deviations (SD) and orthogonal covariates. The simulation results indicate that SCAD and MCP outperform in forecasting and covariate selection with 4 SD (20% and 5% outliers) compared to Autometrics. However, LASSO and AdaLASSO select more covariates than SCAD and MCP and possess higher RMSE. Overall, regularization techniques possess the least RMSE than Autometrics, as Autometrics possesses the least average gauge at the cost of the least average potency. We use COVID-19 cross-sectional data collected from 1 July 2021 to 30 September 2021 for real data analysis. The SCAD and MCP select CRP level, gender, and other comorbidities as an important predictor of hospital stay with the least out-of-sample RMSE of 7.45 and 7.50, respectively.

Robust PC with wild bootstrap estimation of linear model in the presence of outliers, multicollinearity and heteroscedasticity error variance

The regression model estimator is considered efficient if it is robust and resistant to the presence of heteroscedasticity variance, multicollinearity or unusual observations called outliers. However, in regard to these problems, the wild bootstrap and robust wild bootstrap are no longer efficient since they could not produce the smallest variance. Hence this research investigates the use of robust PC with wild bootstrap techniques on regression model as an estimator for real and simulation data in a situation where multicollinearity, heteroscedasticity and multiple outliers are present. This paper proposed a robust procedure based on the weighted residuals which combined the Tukey bisquare weighted function, principal component analysis (PCA) to remedy the multicollinearity problems, least trimmed squares (LTS) estimator, robust location and scale, and the wild bootstrap sampling procedure of Wu and Liu that remedy the heteroscedasticity error variance. RPCW Boot Wu and RPCW Boot Liu were obtained through a modified version of R Boot Wu and R Boot Liu. Finally, based on the real data and simulation study, the performance of the RPCW Boot Wu and RPCW Boot Liu is compared with the existing R Boot Wu, R Boot Liu and also with Boot Wu, Boot Liu using the biased, RMSE and standard error. The numerical example and simulation study shows that the RPCW Boot Wu and RPCW Boot Liu techniques have proven to be a good alternative estimator for regression model with lower standard error values.

On the Efficiency of Modified Regression-type Estimators Using Robust Regression Slopes and Non-conventional Measures of Dispersion

Supplementary variables associated with the study variables have been identified to be helpful in improving the efficiency of ratio, product and regression estimators both at planning and estimation stages. The existing regression-based estimators are functions of regression slopes and known auxiliary variables which are sensitive to outliers. Zaman & Bulut [1] and Zaman [2] addressed the issue of regression slopes in the aforementioned estimators using robust regression slopes like Huber-M, Hampel-M, Least Trimmed Squares (LTS) and Least Absolute Deviation (LAD). However, their estimators still utilized known auxiliary functions which are also sensitive to outliers or extreme values. Similarly, Yadav and Zaman [3] suggested non-conventional robust parameters of auxiliary variable which are robust against outliers. However, the problems of effects of outliers on regression slopes were not considered. In this study, the estimators of Zaman & Bulut [1] and Zaman [2] estimators were modified using robust non-conventional measures and Yadav and Zaman [3] estimators were modified using robust regression slopes. The properties (Biases and MSEs) of the modified estimators were derived up to the first order of approximation using Taylor series approach. The efficiency conditions of the proposed estimators over the existing estimators considered in the study were established. The empirical studies were conducted using stimulation data to investigate the efficiency of the proposed estimators over the efficiency of the existing estimators. The results revealed that the proposed estimators have minimum MSEs and higher PREs among all the competing estimators for the simulated populations in the study. This implies that the proposed estimators are more efficient and can produce better estimate of the population mean compared to other existing estimators considered in the study.

Robust ridge estimator in censored semiparametric linear models

Besides censoring, multicollinearity and outliers are two common problem in regression analysis. In this paper we propose a family of robust ridge estimators for the censored semiparametric regression models. The proposed robust estimators is based on least trimmed squares (LTS) method. This method is insensitive to corruption due to outliers, provided that the outliers constitute less than of the set, in other words, LTS is a robust estimator with a breakdown point. The FAST-LTS algorithm is developed for the computation of the estimators. Furthermore, a robust method for the estimate of shrinkage parameters is suggested. Monté-Carlo simulation study demonstrates the merit of the new method in the aspect of solving the multicollinearity and sensitivity to outliers over the ordinary least squares estimation. Finally, an example of real data is given for illustration.

Partial least trimmed squares regression

Partial least squares (PLS) regression is a linear regression technique and plays an important role in dealing with high-dimensional regressors. Unfortunately, PLS is sensitive to outliers in datasets and consequentially produces a corrupted model. In this paper, we propose a robust method for PLS based on the idea of least trimmed squares (LTS), in which the objective is to minimize the sum of the smallest h squared residuals. However, solving an LTS problem is generally NP-hard. Inspired by the complementary idea of Sim and Hartley, we solve the inverse of the LTS problem instead and formulate it as a concave maximization problem, which is convex and can be solved in polynomial time. Classic PLS as well as two of the most efficient robust PLS methods, Partial Robust M (PRM) regression and RSIMPLS, are compared in this study. Results of both simulation and real data sets show the effectiveness and robustness of our approach.