Heavy-tailed Error Distributions Research Articles

Robust estimation for dynamic single index varying coefficient models

In this article, we propose a new estimation procedure based on modal regression for dynamic single index varying coefficient models, where the index coefficient functions and the link functions are approximated by B-spline basis, respectively. Under some mild regularity conditions, we establish the asymptotic normalities of the obtained spline-based estimators. By introducing an additional tuning parameter (e.g. h), the proposed method can produce robust estimates when the data are not well behaved (e.g. in the presence of outliers and/or heavy-tailed error distributions), and we further calculate the robust estimates by using a modified expectation-maximization-type iteration algorithm, and develop a data driven procedure to select the tuning parameters for the proposed approaches. Finally, some simulations and a real data application are conducted to illustrate the utility of the proposed methodology.

Retire: Robust expectile regression in high dimensions

High-dimensional data can often display heterogeneity due to heteroscedastic variance or inhomogeneous covariate effects. Penalized quantile and expectile regression methods offer useful tools to detect heteroscedasticity in high-dimensional data. The former is computationally challenging due to the non-smooth nature of the check loss, and the latter is sensitive to heavy-tailed error distributions. In this paper, we propose and study (penalized) robust expectile regression (retire), with a focus on iteratively reweighted ℓ1-penalization which reduces the estimation bias from ℓ1-penalization and leads to oracle properties. Theoretically, we establish the statistical properties of the retire estimator under two regimes: (i) low-dimensional regime in which d≪n; (ii) high-dimensional regime in which s≪n≪d with s denoting the number of significant predictors. In the high-dimensional setting, we thoroughly analyze the statistical properties of the solution path of iteratively reweighted ℓ1-penalized retire estimation, adapted from the local linear approximation algorithm for folded-concave regularization. Under a mild minimum signal strength condition, we demonstrate that with as few as log(logd) iterations, the final iterate of our proposed approach achieves the oracle convergence rate. At each iteration, we solve the weighted ℓ1-penalized convex program using a semismooth Newton coordinate descent algorithm. Numerical studies demonstrate the promising performance of the proposed procedure in comparison to both non-robust and quantile regression based alternatives.

Max-sum test based on Spearman's footrule for high-dimensional independence tests

Testing high-dimensional data independence is an essential task of multivariate data analysis in many fields. Typically, the quadratic and extreme value type statistics based on the Pearson correlation coefficient are designed to test dense and sparse alternatives for evaluating high-dimensional independence. However, the two existing popular test methods are sensitive to outliers and are invalid for heavy-tailed error distributions. To overcome these problems, two test statistics, a Spearman's footrule rank-based quadratic scheme and an extreme value type test for dense and sparse alternatives, are proposed, respectively. Under mild conditions, the large sample properties of the resulting test methods are established. Furthermore, the proposed two test statistics are proved to be asymptotically independent. The max-sum test based on Spearman's footrule statistic is developed by combining the proposed quadratic with extreme value statistics, and the asymptotic distribution of the resulting statistical test is established. The simulation results demonstrate that the proposed max-sum test performs well in empirical power and robustness, regardless of whether the data is sparse dependence or not. Finally, to illustrate the use of the proposed test method, two empirical examples of Leaf and Parkinson's disease datasets are provided.

Quantile regression for massive data set

Traditional statistical analysis is challenged by modern massive data sets, which have huge sample size and dimension. Quantile regression has become a popular alternative to least squares method for providing comprehensive description of the response distribution and robustness against heavy-tailed error distributions. On the other hand, non-smooth quantile loss poses a new challenge to massive data sets. To address the problem, we transform the non-differentiable quantile loss function into a convex quadratic loss function based on Expectation-maximization (EM) algorithm using an asymmetric Laplace distribution. Both simulations and real data application are conducted to illustrate the performance of the proposed methods.

Robust variable selection and estimation via adaptive elastic net S-estimators for linear regression

Heavy-tailed error distributions and predictors with anomalous values are ubiquitous in high-dimensional regression problems and can seriously jeopardize the validity of statistical analyses if not properly addressed. For more reliable variable selection and prediction under these adverse conditions, adaptive PENSE, a new robust regularized regression estimator, is proposed. Adaptive PENSE yields reliable variable selection and coefficient estimates even under aberrant contamination in the predictors or residuals. It is shown that the adaptive penalty leads to more robust and reliable variable selection than other penalties, particularly in the presence of gross outliers in the predictor space. It is further demonstrated that adaptive PENSE has strong variable selection properties and that it possesses the oracle property even under heavy-tailed errors and without the need to estimate the error scale. Numerical studies on simulated and real data sets highlight the superior finite-sample performance in a vast range of settings compared to other robust regularized estimators in the case of contaminated samples. An R package implementing a fast algorithm for computing the proposed method and additional simulation results are provided in the supplementary materials.

Robust estimation for varying-coefficient partially linear measurement error model with auxiliary instrumental variables

<abstract><p>We study the varying-coefficient partially linear model when some linear covariates are not observed, but their auxiliary instrumental variables are available. Combining the calibrated error-prone covariates and modal regression, we present a two-stage efficient estimation procedure, which is robust against outliers or heavy-tail error distributions. Asymptotic properties of the resulting estimators are established. Performance of our proposed estimation procedure is illustrated through some numerous simulations and a real example. And the results confirm that the proposed methods are satisfactory.</p></abstract>

ANALYSIS OF GLOBAL AND LOCAL OPTIMA OF REGULARIZED QUANTILE REGRESSION IN HIGH DIMENSIONS: A SUBGRADIENT APPROACH

Regularized quantile regression (QR) is a useful technique for analyzing heterogeneous data under potentially heavy-tailed error contamination in high dimensions. This paper provides a new analysis of the estimation/prediction error bounds of the global solution of $L_1$ -regularized QR (QR-LASSO) and the local solutions of nonconvex regularized QR (QR-NCP) when the number of covariates is greater than the sample size. Our results build upon and significantly generalize the earlier work in the literature. For certain heavy-tailed error distributions and a general class of design matrices, the least-squares-based LASSO cannot achieve the near-oracle rate derived under the normality assumption no matter the choice of the tuning parameter. In contrast, we establish that QR-LASSO achieves the near-oracle estimation error rate for a broad class of models under conditions weaker than those in the literature. For QR-NCP, we establish the novel results that all local optima within a feasible region have desirable estimation accuracy. Our analysis applies to not just the hard sparsity setting commonly used in the literature, but also to the soft sparsity setting which permits many small coefficients. Our approach relies on a unified characterization of the global/local solutions of regularized QR via subgradients using a generalized Karush–Kuhn–Tucker condition. The theory of the paper establishes a key property of the subdifferential of the quantile loss function in high dimensions, which is of independent interest for analyzing other high-dimensional nonsmooth problems.

Robust estimation for varying coefficient partially functional linear regression models based on exponential squared loss function

Abstract In this article, we present a new robust estimation procedure based on the exponential squared loss function for varying coefficient partially functional linear regression models, where the slope function and nonparametric coefficients are approximated by functional principal component basis functions and B splines, respectively. Under some mild conditions, the convergence rates of the resulted estimators are obtained. Simulation studies indicate that our proposed method can achieve robustness against outliers or heavy-tail error distributions and perform no worse than the popular least-squares estimation method for the normal error case. Finally, a real data example is used to illustrate the application of the proposed method.

Robust Asymmetric Safety Kalman Filter for MIMO Radar Resisting Temporary Outliers

This study presents a novel concept of robust Kalman filtering, especially dealing with outliers when detecting human hand intrusion using multiple-input–multiple-output radar. Outliers that severely deviate from the normal value for a certain period have not been sufficiently discussed from the viewpoint of safety and can seriously hamper the performance of the Kalman filter. Therefore, two types of outliers were intensively discussed: the temporary outlier that originated in misdetection and the additive outlier that is generated by heavy-tailed error distribution noise. Moreover, an asymmetric outlier processing strategy was proposed depending on positive and negative outliers. The result shows that the proposed method can resist the additive and temporary outliers.

Testing Equality of Multiple Population Means under Contaminated Normal Model Using the Density Power Divergence.

This paper considers the problem of comparing several means under the one-way Analysis of Variance (ANOVA) setup. In ANOVA, outliers and heavy-tailed error distribution can seriously hinder the treatment effect, leading to false positive or false negative test results. We propose a robust test of ANOVA using an M-estimator based on the density power divergence. Compared with the existing robust and non-robust approaches, the proposed testing procedure is less affected by data contamination and improves the analysis. The asymptotic properties of the proposed test are derived under some regularity conditions. The finite-sample performance of the proposed test is examined via a series of Monte-Carlo experiments and two empirical data examples—bone marrow transplant dataset and glucose level dataset. The results produced by the proposed testing procedure are favorably compared with the classical ANOVA and robust tests based on Huber’s M-estimator and Tukey’s MM-estimator.

Simulating the behavior of a kernel M-estimator for left-truncated and associated model

This paper investigates theoretical and numerical asymptotic properties of a nonparametric M-estimator’s regression function when the data are left truncated and associated. Using specific tools to this weak dependence, we study the strong uniform consistency of the proposed estimator and give its rate of convergence. A large Monte Carlo simulation study is carried out to comfort the good behavior of the M-estimator for different sample sizes and different truncation rates: First we show that the proposed estimator performs better than the Nadaraya-Watson estimator by looking at their respective closeness to the true univariate and bivariate regression function. Thereafter and to highlight the robustness of our estimator, we consider a regression model with a heavy-tailed error distribution and we compare the global mean square error of the two estimators. Finally, we study the behavior of the two estimators in the presence of outliers using the influence function to show the superiority of the M-estimator.

Theoretical Properties of Bayesian Student-$T$ Linear Regression

Student-$t$ linear regression is a commonly used alternative to the normal model in Bayesian analysis when one wants to gain robustness against outliers. The assumption of heavy-tailed error distribution makes the model more adapted to a potential presence of outliers by assigning higher probabilities to extreme values. Even though the Student-$t$ model is often used in practice, not a lot is known about its theoretical properties. In this paper, we aim to fill some gaps by providing analyses in two different asymptotic scenarios. In the first one, outliers are considered to be further and further away from the bulk of the data. The analysis allows to characterize the limiting posterior distribution, a distribution in which a trace of the outliers is present, making the approach partially robust. The impact of the trace is seen to increase with the degrees of freedom of the Student-$t$ distribution assumed. The second asymptotic scenario is one where the sample size increases and the normal model is the true generating process to be able to compare the efficiency of the robust estimator to the ordinary-least-squares one when the latter is the benchmark. The asymptotic efficiency is comparable, in the sense that the variance of the robust estimator is inflated but only by a factor, and this factor converges to 1 as the degrees of freedom increase. The trade-off between robustness and efficiency controlled through the degrees of freedom is thus precisely characterized (at least asymptotically).

Smoothing quantile regression for a distributed system

Quantile regression has become a popular alternative to least squares regression for providing a comprehensive description of the response distribution, and robustness against heavy-tailed error distributions. However, the nonsmooth quantile loss poses new challenges to distributed estimation in both computation and theoretical development. To address this challenge, we use a convolution-type smoothing approach and its Taylor expression to transform the nondifferentiable quantile loss function into a convex quadratic loss function, which admits a fast and scalable algorithm to perform optimization under massive and high-dimensional data. The proposed distributed estimators are both computationally and communication efficient. Moreover, only the gradient information is communicated at each iteration. Theoretically, we show that, after a certain number of iterations, the resulting estimator is statistically as efficient as the global estimator without any restriction on the number of machines. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods.

Development of a new modified hogg type adaptive scheme for multilevel models with diverse error distributions

The rank-based method is a well-known robust estimation technique in analyzing the linear models for non normal error distributions. The efficiency of rank-based analysis can be upgraded by selecting a suitable score function according to the probability distribution of the error term. In this study, a modified version of Hogg’s type adaptive scheme is developed by introducing a new set of cutoff values. The novel idea is to use the rank-based estimation by using the score functions selected through Hogg’s and modified Hogg’s schemes for multilevel models that generate cluster-correlated errors. The efficiency of both schemes is compared for symmetric, asymmetric, and light-tailed to heavy-tailed error distributions under various sample sizes through simulation study based on bias, variance, mean square error, the selected score function, and precision. The modified Hogg’s scheme produces a more efficient rank-based fit than Hogg’s scheme in case of skewed error distribution and produces equal efficiency for symmetric heavy, moderate, and light-tailed distributions. The empirical comparison of score selection through both schemes is also illustrated via a real example. The modified Hogg’s scheme considered the skewness and selected an appropriate score function, giving a more efficient fit than Hogg’s scheme that ignores the skewness.

A Model-Free Approach for Testing Association

Abstract The question of association between outcome and feature is generally framed in the context of a model based on functional and distributional forms. Our motivating application is that of identifying serum biomarkers of angiogenesis, energy metabolism, apoptosis and inflammation, predictive of recurrence after lung resection in node-negative non-small cell lung cancer patients with tumour stage T2a or less. We propose an omnibus approach for testing the association that is free of assumptions on functional forms and distributions and can be used as a general method. This proposed maximal permutation test is based on the idea of thresholding, is readily implementable and is computationally efficient. We demonstrate that the proposed omnibus tests maintain their levels and have strong power for detecting linear, nonlinear and quantile-based associations, even with outlier-prone and heavy-tailed error distributions and under nonparametric setting. We additionally illustrate the use of this approach in model-free feature screening and further examine the level and power of these tests for binary outcome. We compare the performance of the proposed omnibus tests with comparator methods in our motivating application to identify the preoperative serum biomarkers associated with non-small cell lung cancer recurrence in early stage patients.

New quantile based ridge M-estimator for linear regression models with multicollinearity and outliers

The ordinary least squares and ridge regression estimators in a multiple linear regression model with multicollinearity and y-direction outliers lead to unfavorable results. In order to mitigate such situation, the available literature provides few ridge M-estimators to get precise estimates. The ridge parameter, k, plays a vital role in a bias-variance tradeoff for these estimators. However, for high signal-to-noise ratio and multicollinearity with y-direction outliers, the available methods may not perform well in terms of their mean squared error. In this article, we propose a new quantile based ridge M-estimator. The new estimator gives an automated choice of quantile probability of ridge parameter according to the level of noise and multicollinearity. Based on a simulation study, the new estimator outperforms the ordinary least square estimator, ridge estimator, and other considered ridge M-estimators especially for high multicollinearity, significant error variance, and y-direction outliers. Besides normal distribution, new estimator also performs well for heavy-tailed error distribution. Finally, two real-life examples are used to illustrate the application of the proposed estimator.

Robust estimation with modified Huber's function for functional linear models

In this article, we consider a new robust estimation procedure for functional linear models with both slope function and functional predictor approximated by functional principal component basis functions. A modified Huber's function with tail function substituted by the exponential squared loss (ESL) is applied to the estimation procedure for achieving robustness against outliers. The tuning parameters of the new estimation method are data-driven, which enables us to reach better robustness and efficiency than other robust methods in the presence of outliers or heavy-tailed error distribution. We will show that the resulting estimator for the slope function achieves the optimal convergence rate as the least-squares estimator does in the classical functional linear regression. The convergence rate of the prediction in terms of conditional mean squared prediction error is also established. The proposed method is illustrated with simulation studies and a real data example.

Large-scale regression with non-convex loss and penalty

We describe a computational method for parameter estimation in linear regression, that is capable of simultaneously producing sparse estimates and dealing with outliers and heavy-tailed error distributions. The method used is based on the image restoration method proposed in Huang et al. (2017) [13]. It can be applied to problems of arbitrary size. The choice of certain parameters is discussed. Results obtained for simulated and real data are presented.

Quantile LASSO in arbitrage-free option markets

The option price function and the implied volatility surface are both key tools for the derivative pricing strategies and the financial market analysis. Modern and sophisticated methods are used but their credibility suffered due to the financial crisis in 2007–2010. Instead, a method based on a standard semiparametric smoothing is proposed and the overall complexity and robustness (with respect to various anomalies, such as bid-ask spreads, discrete ticks in price, non-synchronous trading, or even heavy tailed error distributions) is achieved by using the conditional quantile estimation. The overestimation and the sparsity principle are adopted to introduce additional flexibility and the LASSO-type penalty and the set of well-defined linear constraints are employed to produce the final estimate which complies with the arbitrage-free criteria dictated by the financial theory. The theoretical results of the model are discussed, finite sample properties are investigated via a simulation study and a practical application of the proposed method is illustrated for the Apple Inc. (AAPL) call options.

Proposing Robust LAD-Atan Penalty of Regression Model Estimation for High Dimensional Data

The issue of penalized regression model has received considerable critical attention to variable selection. It plays an essential role in dealing with high dimensional data. Arctangent denoted by the Atan penalty has been used in both estimation and variable selection as an efficient method recently. However, the Atan penalty is very sensitive to outliers in response to variables or heavy-tailed error distribution. While the least absolute deviation is a good method to get robustness in regression estimation. The specific objective of this research is to propose a robust Atan estimator from combining these two ideas at once. Simulation experiments and real data applications show that the proposed LAD-Atan estimator has superior performance compared with other estimators.