Sufficient Dimension Reduction Methods Research Articles

The ensemble conditional variance estimator for sufficient dimension reduction

Ensemble Conditional Variance Estimation (ECVE) is a novel sufficient dimension reduction (SDR) method in regressions with continuous response and predictors. ECVE applies to general non-additive error regression models and operates under the assumption that the predictors can be replaced by a lower dimensional projection without loss of information. It is a semiparametric forward regression model-based exhaustive sufficient dimension reduction estimation method that is shown to be consistent under mild assumptions. ECVE outperforms central subspace mean average variance estimation (csMAVE), its main competitor, under several simulation settings and in a benchmark data set analysis.

Sparse minimum average variance estimation through signal extraction approach to multivariate regression

In this paper, a new sparse method called (MAVE-SiER) is proposed, to introduce MAVE-SiER, we combined the effective sufficient dimension reduction method MAVE with the sparse method Signal extraction approach to multivariate regression (SiER). MAVE-SiER has the benefit of expanding the Signal extraction method to multivariate regression (SiER) to nonlinear and multi-dimensional regression. MAVE-SiER also allows MAVE to deal with problems which the predictors are highly correlated. MAVE-SiER may estimate dimensions exhaustively while concurrently choosing useful variables. Simulation studies confirmed MAVE-SiER performance.

Sufficient dimension reduction for survival data analysis with error-prone variables

Sufficient dimension reduction (SDR) is an important tool in regression analysis which reduces the dimension of covariates without losing predictive information. Several methods have been proposed to handle data with either censoring in the response or measurement error in covariates. However, little research is available to deal with data having these two features simultaneously. In this paper, we examine this problem. We start with considering the cumulative distribution function in regular settings and propose a valid SDR method to incorporate the effects of censored data and covariates measurement error. Theoretical results are established, and numerical studies are reported to assess the performance of the proposed methods.

Heteroscedasticity testing for semi-parametric multi-index models based on the partial sufficient dimension reduction method

Heteroscedasticity testing is an important issue in regression analysis. In this paper, a model-adaptive test statistic is proposed for the semi-parametric multi-index model. The semi-parametric model studied in this paper includes two types of predictors, namely the predictors of primary interest $X$ and the predictors of secondary interest $W$. In general, the dimension of the variable $X$ is relatively high, leading to the greatly reduced accuracy of the non-parametric estimation.Meanwhile, the existence of the secondary variable $W$ makes the traditional sufficient dimension reduction methods no longer applicable.In this paper, a dimension-reduction test statistic is constructed based on the partial sufficient dimension reduction method, which effectively avoids the estimation difficulties caused by high-dimensional variables. The statistic can be adaptive to many different model structures and thus has good robustness. The asymptotic properties of the proposed test statistic are studied under the null hypothesis and the alternative hypothesis. Simulations and real data examples are given to illustrate the finite sample performance of the test.

Fusing sufficient dimension reduction with neural networks

Neural networks are combined with sufficient dimension reduction methodology in order to remove the limitation of small p and n of the latter. NN-SDR applies when the dependence of the response Y on a set of predictors X is fully captured by the regression function g(B′X), for an unknown function g and low rank parameter B matrix. It is shown that the proposed estimator is on par with competing sufficient dimension reduction methods, such as minimum average variance estimation and conditional variance estimation, in small p and n settings in simulations. Its main advantage is its scalability in regressions with large data, for which the other methods are infeasible.

Surrogate space based dimension reduction for nonignorable nonresponse

Sufficient dimension reduction (SDR) for nonignorable nonresponse poses a challenge and the literature about this issue is very rare. In the nonignorable case, the SDR methods developed for ignorable missing data generally yield serious estimation bias and thus are invalid. A regression-calibration-based cumulative mean estimation (RC-CUME) procedure is proposed to recover the central subspace (CS) with the aid of a surrogate subspace. Asymptotic properties of the RC-CUME are investigated. A modified BIC-type criterion is used to determine the structural dimension of the CS. Some extensions to other SDR methods are presented. Simulation studies are conducted to access the finite-sample performance of the proposed RC-CUME approach, and a real data set is analyzed for illustration.

Comparison of Subspace Dimension Reduction Methods in Logistic Regression

Regression models are very useful in describing and predicting real world phenomena. The Logistic regression is an extremely robust and flexible method for dichotomous classification prediction. This model is a classification model rather than regression model. When the number of predictors in regression models is high, data analysis is difficult. Dimension reduction has become one of the most important issues in regression analysis because of its importance in dealing with problems with high-dimensional data. In this paper, the methods of diminishing the dimension of variables in logistic regression, which include the estimation of central subspace based on the inverse regression, the likelihood acquisition method and principal component analysis are considered. Using a real data associated with the dental problems the Logistic regression is fitted and the correct classification of the data computed. At the end, The simulation study is presented to compare the sufficient dimension reduction methods with each other. In the simulation, MATLAB software is used and the Programs are attached at the end of the article in appendix.

On principal graphical models with application to gene network

The principal graphical model is introduced by incorporating the ideas from the linear sufficient dimension reduction (SDR) methods such as the sliced inverse regression and sliced average variance estimation to a nonparametric graphical model. A nonparametric graphical model is a widely used method to investigate undirected graphs. However, when the number of nodes is large, a nonparametric graphical model suffers from the ‘curse of dimensionality’ because they contain intrinsic high dimensional kernels. The parametric graphical models such as the Gaussian or copula Gaussian graphical models are also well known for their intuitive structure and interpretability. However, they hinge on strong parametric model assumptions. The principal graphical model applies well-known linear SDR techniques to the nonparametric graphical models to enhance performance in high dimensional networks, avoid model assumptions, and maintain interpretability. We use components of linear SDR as modules and implement them in the (p2) pairs of variables in the network to evaluate conditional independence. In the numerical experiment, our methods have competitive accuracy in both low and high-dimensional settings. Our methods are applied to the DREAM 4 challenge gene network dataset and they work well in high dimensional settings with a limited number of observations.

A slice of multivariate dimension reduction

We describe how many dimension reduction strategies are connected conceptually and philosophically, paving the way for a unified approach to multivariate dimension reduction in statistics. Specific methods covered include envelopes, sufficient dimension reduction methods like SIR and SAVE, principal components, principal fitted components, and partial least squares.

Sparse sufficient dimension reduction with heteroscedasticity

Heteroscedasticity often appears in the high-dimensional data analysis. In order to achieve a sparse dimension reduction direction for high-dimensional data with heteroscedasticity, we propose a new sparse sufficient dimension reduction method, called Lasso-PQR. From the candidate matrix derived from the principal quantile regression (PQR) method, we construct a new artificial response variable which is made up from top eigenvectors of the candidate matrix. Then we apply a Lasso regression to obtain sparse dimension reduction directions. While for the “large [Formula: see text] small [Formula: see text]” case that [Formula: see text], we use principal projection to solve the dimension reduction problem in a lower-dimensional subspace and projection back to the original dimension reduction problem. Theoretical properties of the methodology are established. Compared with several existing methods in the simulations and real data analysis, we demonstrate the advantages of our method in the high dimension data with heteroscedasticity.

Feature filter for estimating central mean subspace and its sparse solution

Sufficient dimension reduction, replacing the original predictors with a few linear combinations while keeping all the regression information, has been widely studied. A key goal is to find the central mean subspace, the intersection of all subspaces that provide such a reduction. To this end, a new sufficient dimension reduction method is proposed, with two estimation procedures, through a novel approach of feature filter, applicable to both univariate and multivariate responses. Asymptotic results are established. Estimation methods to determine the structural dimension, to obtain sparse estimator and to deal with large p small n data are provided. The efficacy of the method is demonstrated by simulations and a real data example.

High-Dimensional Elliptical Sliced Inverse Regression in Non-Gaussian Distributions

Sliced inverse regression (SIR) is the most widely used sufficient dimension reduction method due to its simplicity, generality and computational efficiency. However, when the distribution of covariates deviates from multivariate normal distribution, the estimation efficiency of SIR gets rather low, and the SIR estimator may be inconsistent and misleading, especially in the high-dimensional setting. In this article, we propose a robust alternative to SIR—called elliptical sliced inverse regression (ESIR), to analysis high-dimensional, elliptically distributed data. There are wide applications of elliptically distributed data, especially in finance and economics where the distribution of the data is often heavy-tailed. To tackle the heavy-tailed elliptically distributed covariates, we novelly use the multivariate Kendall’s tau matrix in a framework of generalized eigenvalue problem in sufficient dimension reduction. Methodologically, we present a practical algorithm for our method. Theoretically, we investigate the asymptotic behavior of the ESIR estimator under the high-dimensional setting. Extensive simulation results show ESIR significantly improves the estimation efficiency in heavy-tailed scenarios, compared with other robust SIR methods. Analysis of the Istanbul stock exchange dataset also demonstrates the effectiveness of our proposed method. Moreover, ESIR can be easily extended to other sufficient dimension reduction methods and applied to nonelliptical heavy-tailed distributions.

Regularized MAVE through the elastic net with correlated predictors

In this article, we proposed a model-free variable selection method (SMAVE-EN). The concepts of sufficient dimension reduction (SDR) and regularization methods are combined to introduce SMAVE-EN. This method is proposed to produce a shrinkage estimation when the predictors are highly correlated under SDR settings. The advantage of SMAVE-EN is that SMAVE-EN extended Elastic net (EN) to nonlinear and multi-dimensional regression under SDR settings. From another side, the SMAVE-EN enables MAVE to work with problems were the predictors are highly correlated. In addition, SMAVE-EN can exhaustively estimate dimensions, while selecting informative covariates simultaneously under SDR framework. The effectiveness of SMAVE-EN is evaluated by both simulation and real data analysis.

Bayesian model averaging sliced inverse regression

As a popular sufficient dimension reduction method, sliced inverse regression (SIR) (Li, 1991) involves all the predictors. We propose Bayesian model averaging SIR when the central space only involves a subset of the predictors.

Ultrahigh-dimensional sufficient dimension reduction for censored data with measurement error in covariates

In this paper, we consider the ultrahigh-dimensional sufficient dimension reduction (SDR) for censored data and measurement error in covariates. We first propose the feature screening procedure based on censored data and the covariates subject to measurement error. With the suitable correction of mismeasurement, the error-contaminated variables detected by the proposed feature screening procedure are the same as the truly important variables. Based on the selected active variables, we develop the SDR method to estimate the central subspace and the structural dimension with both censored data and measurement error incorporated. The theoretical results of the proposed method are established. Simulation studies are reported to assess the performance of the proposed method. The proposed method is implemented to NKI breast cancer data.

Real-time sufficient dimension reduction through principal least squares support vector machines

We propose a real-time approach for sufficient dimension reduction. Compared with popular sufficient dimension reduction methods including sliced inverse regression and principal support vector machines, the proposed principal least squares support vector machines approach enjoys better estimation of the central subspace. Furthermore, this new proposal can be used in the presence of streamed data for quick real-time updates. It is demonstrated through simulations and real data applications that our proposal performs better and faster than existing algorithms in the literature.

On expectile-assisted inverse regression estimation for sufficient dimension reduction

Moment-based sufficient dimension reduction methods such as sliced inverse regression may not work well in the presence of heteroscedasticity. We propose to first estimate the expectiles through kernel expectile regression, and then carry out dimension reduction based on random projections of the regression expectiles. Several popular inverse regression methods in the literature are extended under this general framework. The proposed expectile-assisted methods outperform existing moment-based dimension reduction methods in both numerical studies and an analysis of the Big Mac data.

Structural reliability assessment through surrogate based importance sampling with dimension reduction

We present a method for reliability assessment in extreme conditions from a numerical simulator through surrogate based importance sampling. As proposed in recent works in the literature, a Kriging surrogate is used to build an approximation of the limit state function and the optimal importance density. Our contribution is then the use of a sufficient dimension reduction method which enables the construction of the limit state function metamodel in lower dimension. The so called augmented failure probability and correction factor are recast in this dimension reduction framework. Simple strategies for metamodel refinement in the dimension reduction subspace are described and, in the case of Gaussian inputs, a computationally efficient MCMC scheme aimed at sampling the quasi-optimal importance density is presented. The case of non-Gaussian inputs is also laid out and it is argued and demonstrated through simulations that this approach can reduce the number of calls to the computer model, which is a crucial factor in reliability analysis. Advantages of this method are also supported by numerical simulations carried on an industrial case study concerned with the extreme response prediction of a wind turbine under wind loading.

MM algorithms for distance covariance based sufficient dimension reduction and sufficient variable selection

Sufficient dimension reduction (SDR) using distance covariance (DCOV) was recently proposed as an approach to dimension-reduction problems. Compared with other SDR methods, it is model-free without estimating link function and does not require any particular distributions on predictors. However, the DCOV-based SDR method involves optimizing a nonsmooth and nonconvex objective function over the Stiefel manifold. To tackle the numerical challenge, the original objective function is equivalently formulated into a DC (Difference of Convex functions) program and an iterative algorithm based on the majorization–minimization (MM) principle is constructed. At each step of the MM algorithm, one iteration of Riemannian Newton’s method is taken to solve the quadratic subproblem on the Stiefel manifold inexactly. In addition, the algorithm can also be readily extended to sufficient variable selection (SVS) using distance covariance. Finally, the convergence property of the proposed algorithm under some regularity conditions is established. Simulation and real data analysis show our algorithm drastically improves the computation efficiency and is robust across various settings compared with the existing method. Matlab codes implementing our methods and scripts for regenerating the numerical results are available at https://github.com/runxiong-wu/MMRN.

A quantile-slicing approach for sufficient dimension reduction with censored responses.

Sufficient dimension reduction (SDR) that effectively reduces the predictor dimension in regression has been popular in high-dimensional data analysis. Under the presence of censoring, however, most existing SDR methods suffer. In this article, we propose a new algorithm to perform SDR with censored responses based on the quantile-slicing scheme recently proposed by Kim etal. First, we estimate the conditional quantile function of the true survival time via the censored kernel quantile regression (Shin etal.) and then slice the data based on the estimated censored regression quantiles instead of the responses. Both simulated and real data analysis demonstrate promising performance of the proposed method.