Finite Sample Breakdown Point Research Articles

Non-asymptotic robustness analysis of regression depth median

The maximum depth estimator (aka depth median) (βRD∗) induced from regression depth (RD) of Rousseeuw and Hubert (1999) is one of the most prevailing estimators in regression. It possesses outstanding robustness similar to the univariate location counterpart. Indeed, βRD∗ can, asymptotically, resist up to 33% contamination without breakdown, in contrast to the 0% for the traditional (least squares and least absolute deviations) estimators (see Van Aelst and Rousseeuw (2000)). The results from Van Aelst and Rousseeuw (2000) are pioneering, yet they are limited to regression-symmetric populations (with a strictly positive density), the ϵ-contamination, maximum-bias model, and in asymptotical sense. With a fixed finite-sample size practice, the most prevailing measure of robustness for estimators is the finite-sample breakdown point (FSBP) (Donoho and Huber, 1983). Despite many attempts made in the literature, only sporadic partial results on FSBP for βRD∗ were obtained whereas an exact FSBP for βRD∗ remained open in the last twenty-plus years. Furthermore, is the asymptotic breakdown value 1/3 (the limit of an increasing sequence of finite-sample breakdown values) relevant in the finite-sample practice? (Or what is the difference between the finite-sample and the limit breakdown values?). Such discussions are yet to be given in the literature. This article addresses the above issues, revealing an intrinsic connection between the regression depth of βRD∗ and the newly obtained exact FSBP. It justifies the employment of βRD∗ as a robust alternative to the traditional estimators and demonstrates the necessity and the merit of using the FSBP in finite-sample real practice.

Robust explicit estimators using the power-weighted repeated medians

This paper consists of two parts. The first part of the paper is to propose an explicit robust estimation method for the regression coefficients in simple linear regression based on the power-weighted repeated medians technique that has a tuning constant for dealing with the trade-offs between efficiency and robustness. We then investigate the lower and upper bounds of the finite-sample breakdown point of the proposed method. The second part of the paper is to show that based on the linearization of the cumulative distribution function, the proposed method can be applied to obtain robust parameter estimators for the Weibull and Birnbaum-Saunders distributions that are commonly used in both reliability and survival analysis. Numerical studies demonstrate that the proposed method performs well in a manner that is approximately comparable with the ordinary least squares method, whereas it is far superior in the presence of data contamination that occurs frequently in practice.

Employing the MCMC technique to compute the projection depth in high dimensions

Projection depth (PD), one of the prevailing location depth notions, is a powerful and favored tool for multivariate nonparametric analysis. It permits the extension of the univariate median and weighted means to a multivariate setting. The multidimensional projection depth median (PM) and depth weighted means, including the Stahel–Donoho (SD) estimator are highly robust and affine equivariant. PM has the highest finite sample breakdown point robustness among affine equivariant location estimators.However, the computation of PD remains a challenge because its exact computation is only feasible for a data set with a dimension that is theoretically no higher than eight but practically no higher than three. Approximate algorithms such as random direction procedure or simulated annealing (SA) algorithm, are time-consuming in high-dimensional cases. Here, we present an efficient SA algorithm and its extension for the computation of PD. Simulated and real data examples indicate that the proposed algorithms outperform their competitors, including the Nelder–Mead method, and the SA algorithm, in high-dimensional cases and can obtain highly accurate results compared with those of the exact algorithm in low-dimensional cases.

Robust subset selection

The best subset selection (or “best subsets”) estimator is a classic tool for sparse regression, and developments in mathematical optimization over the past decade have made it more computationally tractable than ever. Notwithstanding its desirable statistical properties, the best subsets estimator is susceptible to outliers and can break down in the presence of a single contaminated data point. To address this issue, a robust adaption of best subsets is proposed that is highly resistant to contamination in both the response and the predictors. The adapted estimator generalizes the notion of subset selection to both predictors and observations, thereby achieving robustness in addition to sparsity. This procedure, referred to as “robust subset selection” (or “robust subsets”), is defined by a combinatorial optimization problem for which modern discrete optimization methods are applied. The robustness of the estimator in terms of the finite-sample breakdown point of its objective value is formally established. In support of this result, experiments on synthetic and real data are reported that demonstrate the superiority of robust subsets over best subsets in the presence of contamination. Importantly, robust subsets fares competitively across several metrics compared with popular robust adaptions of continuous shrinkage estimators.

Regression: Comparing Predictors and Groups of Predictors Based on a Robust Measure of Association

Let ρj be Pearson’s correlation between Y and Xj (j = 1, 2). A problem that has received considerable attention is testing H0: ρ1 = ρ2. A well-known concern, however, is that Pearson’s correlation is not robust (e.g., Wilcox, 2005), and the usual estimate of ρj , rj has a finite sample breakdown point of only 1/n. The goal in this paper is to consider extensions to situations where Pearson’s correlation is replaced by a particular robust measure of association. Included are results where there are p > 2 predictors and the goal to compare any two subsets of m < p predictors.

On an order-based multivariate median

Although the order-based definition of the univariate median is ubiquitous in statistics, the same order-based definition is typically abandoned when extending the univariate median to higher dimensions. In this paper, an example of order-based multivariate median based on the use of a linear extension of the product order is brought to the attention. Symmetry and internality properties are fulfilled by such order-based multivariate median, however, it turns out that this function fails to fulfil in general other appealing properties such as equivariance to different geometrical transformations, monotonicity properties and continuity. Interestingly, translation and scale equivariance and some monotonicity properties can be guaranteed if the linear extension of the product order is carefully chosen. Finally, it is proved that, unlike in the univariate setting, the finite-sample breakdown point of the order-based multivariate median is 1/n, thus implying that it is not robust in the presence of outliers.

Modal Principal Component Analysis.

Principal component analysis (PCA) is a widely used method for data processing, such as for dimension reduction and visualization. Standard PCA is known to be sensitive to outliers, and various robust PCA methods have been proposed. It has been shown that the robustness of many statistical methods can be improved using mode estimation instead of mean estimation, because mode estimation is not significantly affected by the presence of outliers. Thus, this study proposes a modal principal component analysis (MPCA), which is a robust PCA method based on mode estimation. The proposed method finds the minor component by estimating the mode of the projected data points. As a theoretical contribution, probabilistic convergence property, influence function, finite-sample breakdown point, and its lower bound for the proposed MPCA are derived. The experimental results show that the proposed method has advantages over conventional methods.

Investigation of finite-sample properties of robust location and scale estimators

When the experimental data set is contaminated, we usually employ robust alternatives to common location and scale estimators such as the sample median and Hodges-Lehmann estimators for location and the sample median absolute deviation and Shamos estimators for scale. It is well known that these estimators have high positive asymptotic breakdown points and are Fisher-consistent as the sample size tends to infinity. To the best of our knowledge, the finite-sample properties of these estimators, depending on the sample size, have not well been studied in the literature. In this paper, we fill this gap by providing their closed-form finite-sample breakdown points and calculating the unbiasing factors and relative efficiencies of the robust estimators through the extensive Monte Carlo simulations up to the sample size 100. The numerical study shows that the unbiasing factor improves the finite-sample performance significantly. In addition, we provide the predicted values for the unbiasing factors obtained by using the least squares method which can be used for the case of sample size more than 100.

Robustness of the deepest projection regression functional

Depth notions in regression have been systematically proposed and examined in Zuo (2018). One of the prominent advantages of notion of depth is that it can be directly utilized to introduce median-type deepest estimating functionals (or estimators in empirical distribution case) for location or regression parameters in a multi-dimensional setting. Regression depth shares the advantage. Depth induced deepest estimating functionals are expected to inherit desirable and inherent robustness properties ( e.g. bounded maximum bias and influence function and high breakdown point) as their univariate location counterpart does. Investigating and verifying the robustness of the deepest projection estimating functional (in terms of maximum bias, asymptotic and finite sample breakdown point, and influence function) is the major goal of this article. It turns out that the deepest projection estimating functional possesses a bounded influence function and the best possible asymptotic breakdown point as well as the best finite sample breakdown point with robust choice of its univariate regression and scale component.

On finite-sample robustness of directional location estimators

Robust location estimators for directional data are known for about 30 years. Scientific literature has focused on studying the asymptotic properties of these estimators like consistency and influence function. Apart from the finite-sample breakdown point, the finite-sample performance of robust directional location estimators has attracted less attention. Hence, it is discussed how the finite-sample max-bias of directional location estimators can be evaluated. Additionally, two new robust estimators of the mean direction are introduced: the spherical Minimum Covariance Determinant estimator (sMCD) and the spherical Minimum Spanning Tree estimator (sMST). The sMCD seeks to identify the densest subset of a given size while the sMST seeks for a well-separated subset. Finally, the robust estimators are compared with respect to the max-bias and to the bias under shift outlier scenarios by means of an extensive simulation study. The results indicate that –in contrast to linear data– the maximum likelihood estimator shows high robustness in terms of the finite-sample max-bias. However, robust estimators are clearly superior to the maximum likelihood estimator in shift outlier contamination schemes.

A spatial-type interval-valued median for random intervals

ABSTRACTTo estimate the central tendency or location of a sample of interval-valued data, a standard statistic is the interval-valued sample mean. Its strong sensitivity to outliers or data changes motivates the search for more robust alternatives. In this respect, a more robust location statistic is studied in this paper. This measure is inspired by the concept of spatial median and makes use of the versatile generalized Bertoluzza's metric between intervals, the so-called distance. The problem of minimizing the mean distance to the values the random interval takes, which defines the spatial-type -median, is analysed. Existence and uniqueness of the sample version are shown. Furthermore, the robustness of this proposal is investigated by deriving its finite sample breakdown point. Finally, a real-life example from the Economics field illustrates the robustness of the sample-median, and simulation studies show some comparisons with respect to the mean and several recently introduced robust location measures for interval-valued data.

The Limit of Finite Sample Breakdown Point of Tukey’s Halfspace Median for General Data

Under special conditions on data set and underlying distribution, the limit of finite sample breakdown point of Tukey’s halfspace median (1/3) has been obtained in the literature. In this paper, we establish the result under weaker assumptions imposed on underlying distribution (weak smoothness) and on data set (not necessary in general position). The refined representation of Tukey’s sample depth regions for data set not necessary in general position is also obtained, as a by-product of our derivation.

Efficient robust doubly adaptive regularized regression with applications

We consider the problem of estimation and variable selection for general linear regression models. Regularized regression procedures have been widely used for variable selection, but most existing methods perform poorly in the presence of outliers. We construct a new penalized procedure that simultaneously attains full efficiency and maximum robustness. Furthermore, the proposed procedure satisfies the oracle properties. The new procedure is designed to achieve sparse and robust solutions by imposing adaptive weights on both the decision loss and the penalty function. The proposed method of estimation and variable selection attains full efficiency when the model is correct and, at the same time, achieves maximum robustness when outliers are present. We examine the robustness properties using the finite-sample breakdown point and an influence function. We show that the proposed estimator attains the maximum breakdown point. Furthermore, there is no loss in efficiency when there are no outliers or the error distribution is normal. For practical implementation of the proposed method, we present a computational algorithm. We examine the finite-sample and robustness properties using Monte Carlo studies. Two datasets are also analyzed.

Advantages of M-estimators of location for fuzzy numbers based on Tukey's biweight loss function

M-estimators of location have been adapted to summarize the central tendency of random fuzzy numbers in a robust way. Under mild conditions on the loss function, which include the well-known Huber and Hampel families of loss functions, fuzzy number-valued location M-estimators exist and can be expressed as weighted means of the observations. Huber and Hampel loss functions depend on one and three tuning parameters, respectively. Some empirical analyses have been developed to compare the finite-sample behavior of the corresponding location M-estimators when these tuning parameters are well-chosen quantiles of the distribution of distances from an initial estimate to each observation. In that sense, it has been shown that the flexibility of the three tuning parameters in the Hampel loss function makes the corresponding M-estimator more accurate than the M-estimator based on the Huber loss function in many situations. More recently, Tukey's biweight (or bisquare) loss function has also been used to compute M-estimators of location for random fuzzy numbers. The robustness of all these estimators has been proven by their finite sample breakdown point, but a simulation study to compare the finite-sample behavior of the M-estimators defined in terms of Hampel and Tukey loss functions is still lacking. This paper aims to develop such simulation results and analyze the advantages of choosing the Tukey loss function.

The Effect of Data Contamination in Sliced Inverse Regression and Finite Sample Breakdown Point

Dimension reduction procedures have received increasing consideration over the past decades. Despite this attention, the effect of data contamination or outlying data points in dimension reduction is, however, not well understood, and is compounded by the issue that outliers can be difficult to classify in the presence of many variables. This paper formally investigates the influence of data contamination for sliced inverse regression (SIR), which is a prototypical dimension reduction procedure that targets a lower-dimensional subspace of a set of regressors needed to explain a response variable. We establish a general theory for how estimated reduction subspaces can be distorted through both the number and direction of outlying data points. The results depend critically on the regressor covariance structure and the most harmful types of data contamination are shown to differ in cases where this covariance structure is known or unknown. For example, if the covariance structure is estimated, data contamination is proven to produce an estimated subspace that is automatically orthogonal to the directions of outlying data points, constituting a potentially serious loss of information. Our main results demonstrate the degree to which data contamination indeed causes incorrect dimension reduction, depending on the amount, magnitude, and direction of contamination. Further, by metricizing distances between dimension reduction subspaces, worst case results for data contamination can be formulated to define a finite sample breakdown point for SIR as a measure of global robustness. Our theoretical findings are illustrated through simulation.

Robust variable selection for finite mixture regression models

Finite mixture regression (FMR) models are frequently used in statistical modeling, often with many covariates with low significance. Variable selection techniques can be employed to identify the covariates with little influence on the response. The problem of variable selection in FMR models is studied here. Penalized likelihood-based approaches are sensitive to data contamination, and their efficiency may be significantly reduced when the model is slightly misspecified. We propose a new robust variable selection procedure for FMR models. The proposed method is based on minimum-distance techniques, which seem to have some automatic robustness to model misspecification. We show that the proposed estimator has the variable selection consistency and oracle property. The finite-sample breakdown point of the estimator is established to demonstrate its robustness. We examine small-sample and robustness properties of the estimator using a Monte Carlo study. We also analyze a real data set.

Finite sample breakdown point of Tukey’s halfspace median

Tukey's halfspace median ($\HM$), servicing as the {multivariate} counterpart of the univariate median, has been introduced and extensively studied in the literature. It is supposed and expected to preserve robustness property (the most outstanding property) of the univariate median. One of prevalent quantitative assessments of robustness is finite sample breakdown point (FSBP). Indeed, the FSBP of many multivariate medians have been identified, except for the most prevailing one---the Tukey's halfspace median. This paper presents a precise result on FSBP for Tukey's halfspace median. The result here depicts the complete prospect of the global robustness of $\HM$ in the \emph{finite sample} practical scenario, revealing the dimension effect on the breakdown point robustness and complimenting the existing \emph{asymptotic} breakdown point result.

An Overview on the Statistical Central Tendency for Fuzzy Data Sets

When Statistics deals with data which cannot be expressed in a numerical scale, the scale of fuzzy values (in particular, the scale of fuzzy numbers) often becomes a suitable tool to express such data. In this way, many ratings, opinions, judgements, etc. mostly coming from human valuations can be appropriately described in terms of fuzzy data. To summarize the central tendency of a fuzzy dataset, some measures have been suggested in the literature. This paper aims to review some of the main ones, and examine their properties in a comparative way. A real-life example illustrates their application. Furthermore, the paper shows the statistical robustness (both through the finite sample breakdown point and a simulation study) and the empirical “precision” of the fuzzy number-valued sample measures. Finally, some related developments and future directions are pointed out.

Robust scale estimators for fuzzy data

Observations distant from the majority or deviating from the general pattern often appear in datasets. Classical estimates such as the sample mean or the sample variance can be substantially affected by these observations (outliers). Even a single outlier can have huge distorting influence. However, when one deals with real-valued data there exist robust measures/estimates of location and scale (dispersion) which reduce the influence of these atypical values and provide approximately the same results as the classical estimates applied to the typical data without outliers. In real-life, data to be analyzed and interpreted are not always precisely defined and they cannot be properly expressed by using a numerical scale of measurement. Frequently, some of these imprecise data could be suitably described and modelled by considering a fuzzy rating scale of measurement. In this paper, several well-known scale (dispersion) estimators in the real-valued case are extended for random fuzzy numbers (i.e., random mechanisms generating fuzzy-valued data), and some of their properties as estimators for dispersion are examined. Furthermore, their robust behaviour is analyzed using two powerful tools, namely, the finite sample breakdown point and the sensitivity curves. Simulations, including empirical bias curves, are performed to complete the study.

Minimum volume peeling: A robust nonparametric estimator of the multivariate mode

Among the measures of a distribution’s location, the mode is probably the least often used, although it has some appealing properties. Estimators for the mode of univariate distributions are widely available. However, few contributions can be found for the multivariate case. A consistent direct multivariate mode estimation procedure, called minimum volume peeling, can be outlined as follows. The approach iteratively selects nested subsamples with a decreasing fraction of sample points, looking for the minimum volume subsample at each step. The mode is then estimated by calculating the mean of all points in the final set. The robustness of the method is investigated by analyzing its finite sample breakdown point and algorithms to determine minimum volume sets are discussed. Simulation results confirm that using minimum volume peeling leads to efficient mode estimates both in uncontaminated as well as contaminated situations.