Minimum Average Variance Estimation Research Articles

The Alternating Direction Method of Multipliers for Sufficient Dimension Reduction

The minimum average variance estimation (MAVE) method has proven to be an effective approach to sufficient dimension reduction. In this study, we apply the computationally efficient optimization algorithm named alternating direction method of multipliers (ADMM) to a particular approach (MAVE or minimum average variance estimation) to the problem of sufficient dimension reduction (SDR). Under some assumptions, we prove that the iterative sequence generated by ADMM converges to some point of the associated augmented Lagrangian function. Moreover, that point is stationary. It also presents some numerical simulations on synthetic data to demonstrate the computational efficiency of the algorithm.

Robust Sparse Sufficient Dimension Reduction via Adaptive Lasso Penalty

In some multi-index models applications, there is an important role for dimension reduction and variable selection (VS) methods. The ALMAVE is a method for variables selection under sufficient dimension reduction theory settings. It combines the adaptive Lasso and MAVE (minimum average variance estimation) to produce sparse and accurate solutions. The ALMAVE is a very sensitive method to outliers in the response y due to use the least-squares criterion. In this article, we proposed robust ALMAVE. Also, an efficient estimation algorithm was proposed. The simulation studies and Logo design data analysis were employed to check the effectiveness of ALMAVE.

Robust estimation for partial linear single-index models

In this article, minimum average variance estimation (MAVE) based on local modal regression is proposed for partial linear single-index models, which can be robust to different error distributions or outliers. Asymptotic distributions of the proposed estimators are derived, which have the same convergence rate as the original MAVE based on least squares. A modal EM algorithm is provided to implement our robust estimation. Both simulation studies and a real data example are used to evaluate the finite sample performance of the proposed estimation procedure.

Sparse minimum average variance estimation through signal extraction approach to multivariate regression

In this paper, a new sparse method called (MAVE-SiER) is proposed, to introduce MAVE-SiER, we combined the effective sufficient dimension reduction method MAVE with the sparse method Signal extraction approach to multivariate regression (SiER). MAVE-SiER has the benefit of expanding the Signal extraction method to multivariate regression (SiER) to nonlinear and multi-dimensional regression. MAVE-SiER also allows MAVE to deal with problems which the predictors are highly correlated. MAVE-SiER may estimate dimensions exhaustively while concurrently choosing useful variables. Simulation studies confirmed MAVE-SiER performance.

Fusing sufficient dimension reduction with neural networks

Neural networks are combined with sufficient dimension reduction methodology in order to remove the limitation of small p and n of the latter. NN-SDR applies when the dependence of the response Y on a set of predictors X is fully captured by the regression function g(B′X), for an unknown function g and low rank parameter B matrix. It is shown that the proposed estimator is on par with competing sufficient dimension reduction methods, such as minimum average variance estimation and conditional variance estimation, in small p and n settings in simulations. Its main advantage is its scalability in regressions with large data, for which the other methods are infeasible.

Robust MAVE through nonconvex penalized regression

High dimensionality has been a significant feature in modern statistical modeling. Sufficient dimension reduction (SDR) as an efficient tool aims at reducing the original high dimensional predictors without losing any regression information. Minimum average variance estimation (MAVE) is a popular approach in SDR among others. However, it is not robust to outliers in the response due to the use of least squares. A robust estimation through regularization with case-specific parameters is proposed to achieve robust estimation and outlier detection simultaneously. Under the nonconvex penalized regression framework, two efficient computational strategies are introduced. Simulation studies and a real data application show the efficacy of the proposed approach. Compared with existing methods, the proposed approach is less sensitive to the choice of initial estimators.

Minimum Average Variance Estimation with group Lasso for the multivariate response Central Mean Subspace

The Minimum Average Variance Estimation (MAVE) method and its variants have proven to be effective approaches to the dimension reduction problems. However, as far as we know, using MAVE to estimate the Central Mean Subspace (CMS) with multivariate response receives little attention. This paper proposes a weighted version of MAVE for the CMS with multivariate response. The proposed weighted MAVE method takes account of the correlations among the responses and is a natural extension of the original MAVE method that only targets the univariate response CMS. The algorithm to implement the weighted MAVE method is provided. Asymptotic distribution of the MAVE estimator under the multivariate response setting is also derived. In addition, for the goal of variable selection, we propose incorporating the adaptive group lasso regularization method to induce sparse solutions. We show that the resulting sparse estimator can be calculated in a computationally efficient manner, and the associated BIC-type criterion can consistently select relevant variables. Experimental simulations and a real data analysis demonstrate the effectiveness and usefulness of the proposed methods.

Estimation in single-index varying-coefficient panel data model

In this paper, we study the single-index varying-coefficient panel data model. Combining the refined minimum average variance estimation (RMAVE) method with the local linear regression, we estimate the parameters in single index and link function, and explain the steps of the iterative algorithm. Under certain regularity conditions, the asymptotic properties of the estimators of the parameters and link functions are derived. Finally, numerical simulations are presented and our model is shown to perform better than the single-index panel data model in a real-data example.

Estimation and testing for panel data partially linear single-index models with errors correlated in space and time

We consider a panel data partially linear single-index models (PDPLSIM) with errors correlated in space and time. A serially correlated error structure is adopted for the correlation in time. We propose using a semiparametric minimum average variance estimation (SMAVE) to obtain estimators for both the parameters and unknown link function. We not only establish an asymptotically normal distribution for the estimators of the parameters in the single index and the linear component of the model, but also obtain an asymptotically normal distribution for the nonparametric local linear estimator of the unknown link function. Then, a fitting of spatial and time-wise correlation structures is investigated. Based on the estimators, we propose a generalized F-type test method to deal with testing problems of index parameters of PDPLSIM with errors correlated in space and time. It is shown that under the null hypothesis, the proposed test statistic follows asymptotically a [Formula: see text]-distribution with the scale constant and degrees of freedom being independent of nuisance parameters or functions. Simulated studies and real data examples have been used to illustrate our proposed methodology.

Analysis of panel data partially linear single-index models with serially correlated errors

In this paper, we study a panel data partially linear single-index models (PDPLSIM) with serially correlated errors. A semiparametric minimum average variance estimation method was given to estimate both the unknown parameters and link function. The asymptotic properties of the resulting estimators are established. Then, the serially correlated error structure is investigated. Furthermore, a generalized F-type test method was proposed to check the index parameters. It is shown that under the null hypothesis the proposed test statistic follows asymptotically a χ2-distribution. A numerical simulation and a real data example show that the proposed methods are effective.

On bootstrap consistency of MAVE for single index models

This paper concerns the bootstrap consistency of the minimum average variance estimation (MAVE) method for the single index model. This paper shows that the conditional wild bootstrap estimator of the parameter index shares the same asymptotic covariance of the original MAVE estimator. Thus, the asymptotic distribution can be accurately estimated by the proposed wild bootstrap method. As an application of this method, this paper proposes a conditional Wald type test for the parameter index. It will be shown by simulations that the conditional bootstrap based test is more powerful than the test based on the traditional plug-in covariance estimator. A real data analysis is also provided to demonstrate the effectiveness of the bootstrap method.

Robust dimension reduction using sliced inverse median regression

Dimension reduction is a useful technique when working with high-dimensional predictors, as meaningful data visualizations and graphical analyses using fewer predictors can be achieved. We propose a new non-iterative and robust against extreme values estimation of the effective dimension reduction (e.d.r) subspace, which is based on the estimation of the conditional median function of the predictors given the response. The existing literature on robust estimation of the e.d.r subspace relies on iterative algorithms, such as the composite quantile minimum average variance estimation and the sliced regression. Compared with these existing robust dimension reduction methods, the new method avoids iterations by directly estimating the e.d.r subspace and has better finite sample performance. It is shown that the inverse Tukey and Oja median regression curve falls into the e.d.r subspace, and that its directions can be estimated $$\sqrt{n}$$ -consistently.

Semiparametric jump-preserving estimation for single-index models

ABSTRACTEstimation of the single-index model with a discontinuous unknown link function is considered in this paper. Existed refined minimum average variance estimation (rMAVE) method can estimate the single-index parameter and unknown link function simultaneously by minimising the average pointwise conditional variance, where the conditional variance can be estimated using the local linear fit method with centred kernel function. When there are jumps in the link function, big biases around jumps can appear. For this reason, we embed the jump-preserving technique in the rMAVE method, then propose an adaptive jump-preserving estimation procedure for the single-index model. Concretely speaking, the conditional variance is obtained by the one among local linear fits with centred, left-sided and right-sided kernel functions who has minimum weighted residual mean squares. The resulting estimators can preserve the jumps well and also give smooth estimates of the continuity parts. Asymptotic properties are established under some mild conditions. Simulations and real data analysis show the proposed method works well.

A unified approach to sufficient dimension reduction

Through investigating a recently introduced sufficient dimension reduction method with Hellinger index, this article shows that the generalized Hellinger index unifies three existing dimension reduction methods: kernel discriminant analysis, sliced regression and density minimum average variance estimation, with certain weight functions. The Hellinger index is then extended to regression models with multivariate responses. Furthermore, new algorithms based on Hellinger index to estimate the dual central subspaces and to enable variable selection for sparse models are proposed. Simulation studies and a real data analysis demonstrate the efficacy of the proposed approaches.

Minimum average deviance estimation for sufficient dimension reduction

ABSTRACTSufficient dimension reduction methods aim to reduce the dimensionality of predictors while preserving regression information relevant to the response. In this article, we develop Minimum Average Deviance Estimation (MADE) methodology for sufficient dimension reduction. The purpose of MADE is to generalize Minimum Average Variance Estimation (MAVE) beyond its assumption of additive errors to settings where the outcome follows an exponential family distribution. As in MAVE, a local likelihood approach is used to learn the form of the regression function from the data and the main parameter of interest is a dimension reduction subspace. To estimate this parameter within its natural space, we propose an iterative algorithm where one step utilizes optimization on the Stiefel manifold. MAVE is seen to be a special case of MADE in the case of Gaussian outcomes with a common variance. Several procedures are considered to estimate the reduced dimension and to predict the outcome for an arbitrary covariate value. Initial simulations and data analysis examples yield encouraging results and invite further exploration of the methodology.

Analysis of semi-parametric single-index models by using MAVE-method based on some kernel functions

In this paper, we used many forms of kernel functions with minimum average variance estimation (MAVE) method [Xia2002] we called the proposed methods (MAVE-Biweight), (MAVE-Epanechnikov ) and .(MAVE-Gaussian ) for estimation the parameters and the link function of the single – index model (SIM) comparing with other methods of estimation . to evaluate the performing of the various methods s simulation and a real data have been used, conclusions showed that the (MAVE- Gaussian) method in this paper gave better results compared with other methods depending on the mean squared error (MSE) and mean Absolute error (MAE) criterion for comparison.

Variable selection through adaptive MAVE

Adaptive minimum average variance estimation (MAVE) is an efficient approach for dimension reduction as it can adapt to different error distributions. In this paper, we combine the ideas of adaptive estimation and regression shrinkage, and propose the sparse adaptive MAVE (saMAVE). The saMAVE can estimate the central mean subspace and select informative covariates simultaneously, without assuming any particular model or distribution on the predictor variables. The efficacy of saMAVE is verified through both theoretical results and simulation studies.

On estimating regression-based causal effects using sufficient dimension reduction

SUMMARY In many causal inference problems the parameter of interest is the regression causal effect, defined as the conditional mean difference in the potential outcomes given covariates. In this paper we discuss how sufficient dimension reduction can be used to aid causal inference, and we propose a new estimator of the regression causal effect inspired by minimum average variance estimation. The estimator requires a weaker common support condition than propensity score-based approaches, and can be used to estimate the average causal effect, for which it is shown to be asymptotically super-efficient. Its finite-sample properties are illustrated by simulation.

Robust estimation and variable selection in sufficient dimension reduction

Dimension reduction and variable selection play important roles in high dimensional data analysis. Minimum Average Variance Estimation (MAVE) is an efficient approach among many others. However, because of the use of least squares criterion, MAVE is not robust to outliers in the dependent variable or errors with heavy tailed distributions. A robust extension of MAVE through modal regression is proposed. This new approach can adapt to different error distributions and thus brings robustness to the contamination in the response variable. The estimator is shown to have the same convergence rate as the original MAVE. Furthermore, the proposed method is combined with adaptive LASSO to select informative variables. The efficacy of this new solution is illustrated through simulation studies and a data analysis on Hong Kong air quality.

Semiparametric estimation of the single-index varying-coefficient model

ABSTRACTIn this paper, we consider the choice of pilot estimators for the single-index varying-coefficient model, which may result in radically different estimators, and develop the method for estimating the unknown parameter in this model. To estimate the unknown parameters efficiently, we use the outer product of gradient method to find the consistent initial estimators for interest parameters, and then adopt the refined estimation method to improve the efficiency, which is similar to the refined minimum average variance estimation method. An algorithm is proposed to estimate the model directly. Asymptotic properties for the proposed estimation procedure have been established. The bandwidth selection problem is also considered. Simulation studies are carried out to assess the finite sample performance of the proposed estimators, and efficiency comparisons between the estimation methods are made.