Tukey's Biweight Research Articles

Doubly iteratively reweighted algorithm for constrained compressed sensing models

We propose a new algorithmic framework for constrained compressed sensing models that admit nonconvex sparsity-inducing regularizers including the log-penalty function as objectives, and nonconvex loss functions such as the Cauchy loss function and the Tukey biweight loss function in the constraint. Our framework employs iteratively reweighted $$\ell _1$$ and $$\ell _2$$ schemes to construct subproblems that can be efficiently solved by well-developed solvers for basis pursuit denoising such as SPGL1 by van den Berg and Friedlander (SIAM J Sci Comput 31:890-912, 2008). We propose a new termination criterion for the subproblem solvers that allows them to return an infeasible solution, with a suitably constructed feasible point satisfying a descent condition. The feasible point construction step is the key for establishing the well-definedness of our proposed algorithm, and we also prove that any accumulation point of this sequence of feasible points is a stationary point of the constrained compressed sensing model, under suitable assumptions. Finally, we compare numerically our algorithm (with subproblems solved by SPGL1 or the alternating direction method of multipliers) against the SCP $$_\textrm{ls}$$ in Yu et al. (SIAM J Optim 31: 2024-2054, 2021) on solving constrained compressed sensing models with the log-penalty function as the objective and the Cauchy loss function in the constraint, for badly scaled measurement matrices. Our computational results show that our approaches return solutions with better recovery errors, and are always faster.

Robust Group Identification and Variable Selection in Sliced Inverse Regression Using Tukey's Biweight Criterion and Ball Covariance

The SSIR-PACS is a group identification and a model-free variable selection method under sufficient dimension reduction (SDR) settings. It combined the Pairwise Absolute Clustering and Sparsity (PACS) with sliced inverse regression (SIR) methods to produce solutions with sparsity and the ability of group identification. However, the SSIR-PACS depends on classical estimates for dispersion and location, squared loss function, and non-robust weights for outliers. In this paper, a robust version of SSIR-PACS (RSSIR-PACS) is proposed. We replaced the squared loss by the criterion of Tukey's biweight. Also, the non-robust weights to outliers, which depend on Pearson’s correlations, are substituted with robust weights based on recently developed ball correlation. Moreover, the estimates of the mean and covariance matrix are substituted by the median and ball covariance, respectively. The RSSIR-PACS is robust to outliers in both the response and covariates. According to the results of simulations, RSSIR-PACS produces very good results. If the outliers are existing, the efficacy of RSSIR-PACS is considerably better than the efficacy of the competitors. In addition, a robust criteria to estimate the structural dimension d is proposed. The RSSIR-PACS makes SSIR-PACS practically feasible. Also, we employed real data to demonstrate the utility of RSSIR-PACS.

Robust Geodesic Regression

This paper studies robust regression for data on Riemannian manifolds. Geodesic regression is the generalization of linear regression to a setting with a manifold-valued dependent variable and one or more real-valued independent variables. The existing work on geodesic regression uses the sum-of-squared errors to find the solution, but as in the classical Euclidean case, the least-squares method is highly sensitive to outliers. In this paper, we use M-type estimators, including the \(L_1\), Huber and Tukey biweight estimators, to perform robust geodesic regression, and describe how to calculate the tuning parameters for the latter two. We show that, on compact symmetric spaces, all M-type estimators are maximum likelihood estimators, and argue in favor of a general preference for the \(L_1\) estimator over the \(L_2\) and Huber estimators on high-dimensional spaces. A derivation of the Riemannian normal distribution on \(S^n\) and \(\mathbb {H}^n\) is also included. Results from numerical examples, including analysis of real neuroimaging data, demonstrate the promising empirical properties of the proposed approach.

Certifiably Optimal Outlier-Robust Geometric Perception: Semidefinite Relaxations and Scalable Global Optimization.

We propose the first general and scalable framework to design certifiable algorithms for robust geometric perception in the presence of outliers. Our first contribution is to show that estimation using common robust costs, such as truncated least squares (TLS), maximum consensus, Geman-McClure, Tukey's biweight, among others, can be reformulated as polynomial optimization problems (POPs). By focusing on the TLS cost, our second contribution is to exploit sparsity in the POP and propose a sparse semidefinite programming (SDP) relaxation that is much smaller than the standard Lasserre's hierarchy while preserving empirical exactness, i.e., the SDP recovers the optimizer of the nonconvex POP with an optimality certificate. Our third contribution is to solve the SDP relaxations at an unprecedented scale and accuracy by presenting [Formula: see text], a solver that blends global descent on the convex SDP with fast local search on the nonconvex POP. Our fourth contribution is an evaluation of the proposed framework on six geometric perception problems including single and multiple rotation averaging, point cloud and mesh registration, absolute pose estimation, and category-level object pose and shape estimation. Our experiments demonstrate that (i) our sparse SDP relaxation is empirically exact with up to 60%- 90% outliers across applications; (ii) while still being far from real-time, [Formula: see text] is up to 100 times faster than existing SDP solvers on medium-scale problems, and is the only solver that can solve large-scale SDPs with hundreds of thousands of constraints to high accuracy; (iii) [Formula: see text] safeguards existing fast heuristics for robust estimation (e.g., [Formula: see text] or Graduated Non-Convexity), i.e., it certifies global optimality if the heuristic estimates are optimal, or detects and allows escaping local optima when the heuristic estimates are suboptimal.

A New Efficient Redescending M-Estimator for Robust Fitting of Linear Regression Models in the Presence of Outliers

Robust regression is an important iterative procedure that seeks analyzing data sets that are contaminated with outliers and unusual observations and reducing their impact over regression coefficients. Robust estimation methods have been introduced to deal with the problem of outliers and provide efficient and stable estimates in their presence. Various robust estimators have been developed in the literature to restrict the unbounded influence of the outliers or leverage points on the model estimates. Here, a new redescending M-estimator is proposed using a novel objective function with the prime focus on getting highly robust and efficient estimates that give promising results. It is evident from the results that, for normal and clean data, the proposed estimator is almost as efficient as ordinary least square method and, however, becomes highly resistant to outliers when it is used for contaminated datasets. The simulation study is being carried out to assess the performance of the proposed redescending M-estimator over different data generation scenarios including normal, t-distribution, and double exponential distributions with different levels of outliers’ contamination, and the results are compared with the existing redescending M-estimators, e.g., Huber, Tukey Biweight, Hampel, and Andrew-Sign function. The performance of the proposed estimators was also checked using real-life data applications of the estimators and found that the proposed estimators give promising results as compared to the existing estimators.

A RPCA-Based Tukey's Biweight for Clustering Identification on Extreme Rainfall Data

In high dimensional data, Principal Component Analysis (PCA)-based Pearson correlation remains broadly employed to reduce the data dimensions and to improve the effectiveness of the clustering partitions. Besides being prone to sensitivity on non-Gaussian distributed data, in a high dimensional data analysis, this algorithm may influence the partitions of cluster as well as generate exceptionally imbalanced clusters due to its assigned equal weight to each observation pairs. To solve the unbalanced clusters in hydrological study caused by skewed character of the dataset, this study came out with a robust method of PCA in term of the correlation. This study will explain a RPCA to be proposed as an alternative to classical PCA in reducing high dimensional dataset to a lower form as well as obtain balance clustering result. This study improved where RPCA managed to downweigh the far-from-center outliers and develop the cluster partitions. The results for both methods are compared in term of number of components and clusters obtained as well as the clustering validity. Regarding the internal and stability validation criteria, this study focuses on the cluster's quality in order to validate the results of clusters obtained for both methods. From the findings, the amount of clusters had improved significantly by using RPCA compared to classical PCA. This proved that the proposed approach are outliers resistant than classical PCA as the proposed approach made a thorough observation assessment and downweigh the ones which were distant from the data center.

A Framework of Learning Through Empirical Gain Maximization.

We develop in this letter a framework of empirical gain maximization (EGM) to address the robust regression problem where heavy-tailed noise or outliers may be present in the response variable. The idea of EGM is to approximate the density function of the noise distribution instead of approximating the truth function directly as usual. Unlike the classical maximum likelihood estimation that encourages equal importance of all observations and could be problematic in the presence of abnormal observations, EGM schemes can be interpreted from a minimum distance estimation viewpoint and allow the ignorance of those observations. Furthermore, we show that several well-known robust nonconvex regression paradigms, such as Tukey regression and truncated least square regression, can be reformulated into this new framework. We then develop a learning theory for EGM by means of which a unified analysis can be conducted for these well-established but not fully understood regression approaches. This new framework leads to a novel interpretation of existing bounded nonconvex loss functions. Within this new framework, the two seemingly irrelevant terminologies, the well-known Tukey's biweight loss for robust regression and the triweight kernel for nonparametric smoothing, are closely related. More precisely, we show that Tukey's biweight loss can be derived from the triweight kernel. Other frequently employed bounded nonconvex loss functions in machine learning, such as the truncated square loss, the Geman-McClure loss, and the exponential squared loss, can also be reformulated from certain smoothing kernels in statistics. In addition, the new framework enables us to devise new bounded nonconvex loss functions for robust learning.

Robust regression with asymmetric loss functions.

In robust regression, it is usually assumed that the distribution of the error term is symmetric or the data are symmetrically contaminated by outliers. However, this assumption is usually not satisfied in practical problems, and thus if the traditional robust methods, such as Tukey's biweight and Huber's method, are used to estimate the regression parameters, the efficiency of the parameter estimation can be lost. In this paper, we construct an asymmetric Tukey's biweight loss function with two tuning parameters and propose a data-driven method to find the most appropriate tuning parameters. Furthermore, we provide an adaptive algorithm to obtain robust and efficient parameter estimates. Our extensive simulation studies suggest that the proposed method performs better than the symmetric methods when error terms follow an asymmetric distribution or are asymmetrically contaminated. Finally, a cardiovascular risk factors dataset is analyzed to illustrate the proposed method.

Robust Estimation for a Generalised Ratio Model

It is known that data such as business sales and household income need data transformation prior to regression estimate as the data has a homoscedastic error. However, data transformations make the estimation of mean and total unstable. Therefore, the ratio model is often used for imputation in the field of official statistics to avoid the problem. Our study aims to robustify the estimator following the ratio model by means of M-estimation. Reformulation of the conventional ratio model with homoscedastic quasi-error term provides quasi-residuals which can be used as a measure of outlyingness as same as a linear regression model. A generalisation of the model, which accommodates varied error terms with different heteroscedasticity, is also proposed. Functions for robustified estimators of the generalised ratio model are implemented by the iterative re-weighted least squares algorithm in R environment and illustrated using random datasets. Monte Carlo simulation confirms accuracy of the proposed estimators, as well as their computational efficiency. A comparison of the scale parameters between the average absolute deviation (AAD) and median absolute deviation (MAD) is made regarding Tukey's biweight function. The results with Huber's weight function are also provided for reference. The proposed robust estimator of the generalised ratio model is used for imputation of major corporate accounting items of the 2016 Economic Census for Business Activity in Japan.

Tukey’s biweight estimation for uncertain regression model with imprecise observations

The purpose of regression analysis is to study how a response variable has a relation to a vector of explanatory variables. Traditionally, statisticians assume that the observation data are precise, and we can get some exact values. However, in many cases, the imprecise observation data are available. We assume that these data are uncertain variables in the sense of uncertainty theory. In this paper, the Tukey biweight or bisquare family of loss functions is applied to estimate unknown parameters satisfying the uncertain regression model. First, the Tukey biweight estimations of three types of regression models are given, namely linear, asymptotic and Michaelis–Menten. Then an empirical study is presented to verify the feasibility of this approach. Finally, the effectiveness of this method in weakening the outliers influence is shown by the comparative analysis.

Wōtan: Comprehensive Time-series Detrending in Python

The detection of transiting exoplanets in time-series photometry requires the removal or modeling of instrumental and stellar noise. While instrumental systematics can be reduced using methods such as pixel level decorrelation, removing stellar trends while preserving transit signals proves challenging. Due to vast archives of light curves from recent transit surveys, there is a strong need for accurate automatic detrending, without human intervention. A large variety of detrending algorithms are in active use, but their comparative performance for transit discovery is unexplored. We benchmark all commonly used detrending methods against hundreds of Kepler, K2, and TESS planets, selected to represent the most difficult cases for systems with small planet-to-star radius ratios. The full parameter range is explored for each method to determine the best choices for planet discovery. We conclude that the ideal method is a time-windowed slider with an iterative robust location estimator based on Tukey's biweight. This method recovers 99% and 94% of the shallowest Kepler and K2 planets, respectively. We include an additional analysis for young stars with extreme variability and conclude they are best treated using a spline-based method with a robust Huber estimator. All stellar detrending methods explored are available for public use in wotan, an open-source Python package on GitHub (see https://github.com/hippke/wotan).

Robust algorithms for multiphase regression models

This paper proposes a robust procedure for solving multiphase regression problems that is efficient enough to deal with data contaminated by atypical observations due to measurement errors or those drawn from heavy-tailed distributions. Incorporating the expectation and maximization algorithm with the M-estimation technique, we simultaneously derive robust estimates of the change-points and regression parameters, yet as the proposed method is still not resistant to high leverage outliers we further suggest a modified version by first moderately trimming those outliers and then implementing the new procedure for the trimmed data. This study sets up two robust algorithms using the Huber loss function and Tukey's biweight function to respectively replace the least squares criterion in the normality-based expectation and maximization algorithm, illustrating the effectiveness and superiority of the proposed algorithms through extensive simulations and sensitivity analyses. Experimental results show the ability of the proposed method to withstand outliers and heavy-tailed distributions. Moreover, as resistance to high leverage outliers is particularly important due to their devastating effect on fitting a regression model to data, various real-world applications show the practicability of this approach.

Video despeckling using Shearlet tensor-based anisotropic diffusion

This paper provides an effective method for video speckle noise reduction based on 3D Tensor-based Anisotropic Diffusion technique in the Shearlet domain (V-STAD). The proposed model exploits the multi-scale geometric representation and the sparsity property of the shearlet transform to apply a robust tensorial diffusion at each shearlet coefficients. In fact, the robustness of diffusion tensor image smoothing was not well investigated. Therefore, we adopted the robust Tukey's biweight function in the proposed tensor-based anisotropic diffusion. By this way, the filter benefits from robust statistics and sparse directional image representation property of the shearlet transform in addition to the intrinsic temporal correlations between frames to be adopted to the anisotropic nature of diffusion tensor. The experimental results demonstrate promising despeckling solution as compared to well-known state-of-the-art video denoising methods, like VBM3D and VIDOLSAT. The proposed method has clearly shown superior despeckling capability and it simultaneously demonstrated better local image structures preservation without introducing artifacts.

An Extra-Rate Spatial Enhancement Constructed by MSRR using Regularized Technique and SSRR using High-Frequency Pre-Forecasting

Under many circumstances, a high spatial resolution image (HR) is greatly needed for modern applications nevertheless the HR image captured device is usually overpriced cost. Hence, Super Resolution Reconstruction (SRR) technique, which can reconstruct a HR image from a single LR image or many LR images by using algebraic formulation, is one of the modern research fields in digital image processing (DIP) and Computer Vision (CV). In this paper, an extra-rate spatial enhancement constructed by MSRR (Multi-frame Super Resolution Reconstruction) using regularized technique and SSRR (Single-frame Super Resolution Reconstruction) using high-frequency pre-forecasting is presented in order to enlarge up to 16x ratio rate. Initially, a group of captured images with low spatial resolution are mathematically fused by MSRR using regularized technique established on a recursive Maximum Likelihood and Tukey's Biweight norm in order to enlarge up to 4x ratio rate. Next, this 4x enhanced image is enlarged to be 16x spatial resolution image by SSRR established on the high-frequency pre-forecasting. In the verification experimentation section, the verification outcome demonstrates that the proposed spatial enhancement is successful for enlarging HR image with 16x ratio rate with finer quality.

Advantages of M-estimators of location for fuzzy numbers based on Tukey's biweight loss function

M-estimators of location have been adapted to summarize the central tendency of random fuzzy numbers in a robust way. Under mild conditions on the loss function, which include the well-known Huber and Hampel families of loss functions, fuzzy number-valued location M-estimators exist and can be expressed as weighted means of the observations. Huber and Hampel loss functions depend on one and three tuning parameters, respectively. Some empirical analyses have been developed to compare the finite-sample behavior of the corresponding location M-estimators when these tuning parameters are well-chosen quantiles of the distribution of distances from an initial estimate to each observation. In that sense, it has been shown that the flexibility of the three tuning parameters in the Hampel loss function makes the corresponding M-estimator more accurate than the M-estimator based on the Huber loss function in many situations. More recently, Tukey's biweight (or bisquare) loss function has also been used to compute M-estimators of location for random fuzzy numbers. The robustness of all these estimators has been proven by their finite sample breakdown point, but a simulation study to compare the finite-sample behavior of the M-estimators defined in terms of Hampel and Tukey loss functions is still lacking. This paper aims to develop such simulation results and analyze the advantages of choosing the Tukey loss function.

Robust Lasso Regression Using Tukey's Biweight Criterion

ABSTRACTThe adaptive lasso is a method for performing simultaneous parameter estimation and variable selection. The adaptive weights used in its penalty term mean that the adaptive lasso achieves the oracle property. In this work, we propose an extension of the adaptive lasso named the Tukey-lasso. By using Tukey's biweight criterion, instead of squared loss, the Tukey-lasso is resistant to outliers in both the response and covariates. Importantly, we demonstrate that the Tukey-lasso also enjoys the oracle property. A fast accelerated proximal gradient (APG) algorithm is proposed and implemented for computing the Tukey-lasso. Our extensive simulations show that the Tukey-lasso, implemented with the APG algorithm, achieves very reliable results, including for high-dimensional data where p > n. In the presence of outliers, the Tukey-lasso is shown to offer substantial improvements in performance compared to the adaptive lasso and other robust implementations of the lasso. Real-data examples further demonstrate the utility of the Tukey-lasso. Supplementary materials for this article are available online.

MR image intensity inhomogeneity correction

MR technology is one of the best and most reliable ways of studying the brain. Its main drawback is the so-called intensity inhomogeneity or bias field which impairs the visual inspection and the medical proceedings for diagnosis and strongly affects the quantitative image analysis. Noise is yet another artifact in medical images. In order to accurately and effectively restore the original signal, reference is hereof made to filtering, bias correction and quantitative analysis of correction. In this report, two denoising algorithms are used; (i) Basis rotation fields of experts (BRFoE) and (ii) Anisotropic Diffusion (when Gaussian noise, the Perona-Malik and Tukey's biweight functions and the standard deviation of the noise of the input image are considered).

Monitoring robust regression

Robust methods are little applied (although much studied by statisticians). We monitor very robust regression by looking at the be- haviour of residuals and test statistics as we smoothly change the robustness of parameter estimation from a breakdown point of 50% to non-robust least squares. The resulting procedure provides insight into the structure of the data including outliers and the presence of more than one population. Moni- toring overcomes the hindrances to the routine adoption of robust methods, being informative about the choice between the various robust procedures. Methods tuned to give nominal high efficiency fail with our most compli- cated example. We find that the most informative analyses come from S estimates combined with Tukey's biweight or with the optimalfunctions. For our major example with 1,949 observations and 13 explanatory vari- ables, we combine robust S estimation with regression using the forward search, so obtaining an understanding of the importance of individual obser- vations, which is missing from standard robust procedures. We discover that the data come from two different populations. They also contain six outliers. Our analyses are accompanied by numerous graphs. Algebraic results are contained in two appendices, the second of which provides useful new results on the absolute odd moments of elliptically truncated multivariate normal random variables.

Improved Cluster Partition in Principal Component Analysis Guided Clustering

Principal component analysis (PCA) guided clustering approach is widely used in high dimensional data to improve the efficiency of K- means cluster solutions. Typically, Pearson correlation is used in PCA to provide an eigen-analysis to obtain the associated components that account for most of the variations in the data. However, PCA based Pearson correlation can be sensitive on non-Gaussian distributed data, which involve skewed observations such as outlying values. Thus, applying PCA based Pearson correlation on such data could affect cluster partitions and generate extremely imbalanced clusters in a high dimensional space. In this study, Tukey's biweight correlation based on M-estimate approach in PCA is used as an alternative to Pearson correlation. This approach is more resistant to outlying values as it examines each observation and down weight those that lie far from the center of the data. In particular two major features are highlighted: (1) fewer components are retained and imbalanced clusters at the recommended cumulative percentage of variation threshold is avoided; (2) the cluster quality with respect to external, internal and relative criteria as shown in Rand, Silhouette and Davies-Bouldin indices, outperform that of the clusters from PCA based Pearson correlation.

An evaluation of two-channel ChIP-on-chip and DNA methylation microarray normalization strategies

BackgroundThe combination of chromatin immunoprecipitation with two-channel microarray technology enables genome-wide mapping of binding sites of DNA-interacting proteins (ChIP-on-chip) or sites with methylated CpG di-nucleotides (DNA methylation microarray). These powerful tools are the gateway to understanding gene transcription regulation. Since the goals of such studies, the sample preparation procedures, the microarray content and study design are all different from transcriptomics microarrays, the data pre-processing strategies traditionally applied to transcriptomics microarrays may not be appropriate. Particularly, the main challenge of the normalization of "regulation microarrays" is (i) to make the data of individual microarrays quantitatively comparable and (ii) to keep the signals of the enriched probes, representing DNA sequences from the precipitate, as distinguishable as possible from the signals of the un-enriched probes, representing DNA sequences largely absent from the precipitate.ResultsWe compare several widely used normalization approaches (VSN, LOWESS, quantile, T-quantile, Tukey's biweight scaling, Peng's method) applied to a selection of regulation microarray datasets, ranging from DNA methylation to transcription factor binding and histone modification studies. Through comparison of the data distributions of control probes and gene promoter probes before and after normalization, and assessment of the power to identify known enriched genomic regions after normalization, we demonstrate that there are clear differences in performance between normalization procedures.ConclusionT-quantile normalization applied separately on the channels and Tukey's biweight scaling outperform other methods in terms of the conservation of enriched and un-enriched signal separation, as well as in identification of genomic regions known to be enriched. T-quantile normalization is preferable as it additionally improves comparability between microarrays. In contrast, popular normalization approaches like quantile, LOWESS, Peng's method and VSN normalization alter the data distributions of regulation microarrays to such an extent that using these approaches will impact the reliability of the downstream analysis substantially.