Sufficient Dimension Reduction Methodology Research Articles

Sufficient Dimension Reduction: An Information-Theoretic Viewpoint.

There has been a lot of interest in sufficient dimension reduction (SDR) methodologies, as well as nonlinear extensions in the statistics literature. The SDR methodology has previously been motivated by several considerations: (a) finding data-driven subspaces that capture the essential facets of regression relationships; (b) analyzing data in a ‘model-free’ manner. In this article, we develop an approach to interpreting SDR techniques using information theory. Such a framework leads to a more assumption-lean understanding of what SDR methods do and also allows for some connections to results in the information theory literature.

Fusing sufficient dimension reduction with neural networks

Neural networks are combined with sufficient dimension reduction methodology in order to remove the limitation of small p and n of the latter. NN-SDR applies when the dependence of the response Y on a set of predictors X is fully captured by the regression function g(B′X), for an unknown function g and low rank parameter B matrix. It is shown that the proposed estimator is on par with competing sufficient dimension reduction methods, such as minimum average variance estimation and conditional variance estimation, in small p and n settings in simulations. Its main advantage is its scalability in regressions with large data, for which the other methods are infeasible.

On hierarchical clustering in sufficient dimension reduction

The K-means clustering algorithm has had successful application in sufficient dimension reduction. Unfortunately, the algorithm does have reproducibility and nestness, which will be discussed in this paper. These are clear deficits for the K-means clustering algorithm; however, the hierarchical clustering algorithm has both reproducibility and nestness, but intensive comparison between K-means and hierarchical clustering algorithm has not yet been done in a sufficient dimension reduction context. In this paper, we rigorously study the two clustering algorithms for two popular sufficient dimension reduction methodology of inverse mean and clustering mean methods throughout intensive numerical studies. Simulation studies and two real data examples confirm that the use of hierarchical clustering algorithm has a potential advantage over the K-means algorithm.

Fused clustering mean estimation of central subspace

Recently, Yoo (Statistics 50:1086–1099, 2016) newly defines an informative predictor subspace to contain the central subspace. The method to estimate the informative predictor subspace does not require any of the conditions assumed to hold in usual sufficient dimension reduction methodologies. However, like sliced inverse regression (Li in J Am Stat Assoc 86:316–342, 1991) and sliced average variance estimation (Cook and Weisberg in J Am Stat Assoc 86:328–332, 1991), its non-asymptotic behavior in the estimation is sensitive to the choices of the categorization of the predictors and response. The paper develops an estimation approach that is robust to the categorization choices. For this, sample kernel matrices are combined in two ways. Numerical studies and real data analysis are presented to confirm the potential usefulness of the proposed approach in practice.

Analysis of microarray right-censored data through fused sliced inverse regression

Sufficient dimension reduction (SDR) for a regression pursue a replacement of the original p-dimensional predictors with its lower-dimensional linear projection. The so-called sliced inverse regression (SIR; [5]) arguably has the longest history in SDR methodologies, but it is still one of the most popular one. The SIR is known to be easily affected by the number of slices, which is one of its critical deficits. Recently, a fused approach for SIR is proposed to relieve this weakness, which fuses the kernel matrices computed by the SIR application from various numbers of slices. In the paper, the fused SIR is applied to a large-p-small n regression of a high-dimensional microarray right-censored data to show its practical advantage over usual SIR application. Through model validation, it is confirmed that the fused SIR outperforms the SIR with any number of slices under consideration.

Tutorial: Methodologies for sufficient dimension reduction in regression

In the paper, as a sequence of the first tutorial, we discuss sufficient dimension reduction methodologies used to estimate central subspace (sliced inverse regression, sliced average variance estimation), central mean subspace (ordinary least square, principal Hessian direction, iterative Hessian transformation), and central k th moment subspace (covariance method). Large-sample tests to determine the structural dimensions of the three target subspaces are well derived in most of the methodologies; however, a permutation test (which does not require large-sample distributions) is introduced. The test can be applied to the methodologies discussed in the paper. Theoretical relationships among the sufficient dimension reduction methodologies are also investigated and real data analysis is presented for illustration purposes. A seeded dimension reduction approach is then introduced for the methodologies to apply to large p small n regressions.

Group-wise sufficient dimension reduction with principal fitted components

Sufficient dimension reduction methodologies in regressions of Y on a p-variate X aim at obtaining a reduction $$R(X) \in {\mathbb R}^{d}, d \le p$$R(X)źRd,d≤p, that retains all the regression information of Y in X. When the predictors fall naturally into a number of known groups or domains, it has been established that exploiting the grouping information often leads to more effective sufficient dimension reduction of the predictors. In this article, we consider group-wise sufficient dimension reduction based on principal fitted components, when the grouping information is unknown. Principal fitted components methodology is coupled with an agglomerative clustering procedure to identify a suitable grouping structure. Simulations and real data analysis demonstrate that the group-wise principal fitted components sufficient dimension reduction is superior to the standard principal fitted components and to general sufficient dimension reduction methods.

A Note on Bootstrapping in Sufficient Dimension Reduction

A permutation test is the popular and attractive alternative to derive asymptotic distributions of dimension test statistics in sufficient dimension reduction methodologies; however, recent studies show that a bootstrapping technique also can be used. We consider two types of bootstrapping dimension determination, which are partial and whole bootstrapping procedures. Numerical studies compare the permutation test and the two bootstrapping procedures; subsequently, real data application is presented. Considering two additional bootstrapping procedures to the existing permutation test, one has more supporting evidence for the dimension estimation of the central subspace that allow it to be determined more convincingly.

Ldr: AnRSoftware Package for Likelihood-Based Sufficient Dimension Reduction

In regression settings, a sufficient dimension reduction (SDR) method seeks the core information in a p-vector predictor that completely captures its relationship with a response. The reduced predictor may reside in a lower dimension d < p, improving ability to visualize data and predict future observations, and mitigating dimensionality issues when carrying out further analysis. We introduce ldr, a new R software package that implements three recently proposed likelihood-based methods for SDR: covariance reduction, likelihood acquired directions, and principal fitted components. All three methods reduce the dimensionality of the data by pro jection into lower dimensional subspaces. The package also implements a variable screening method built upon principal fitted components which makes use of flexible basis functions to capture the dependencies between the predictors and the response. Examples are given to demonstrate likelihood-based SDR analyses using ldr, including estimation of the dimension of reduction subspaces and selection of basis functions. The ldr package provides a framework that we hope to grow into a comprehensive library of likelihood-based SDR methodologies.

On determining the structural dimension via directional regression

Specifying the structural dimension is an important first step for the sufficient dimension reduction methodology. Based on the popular sequential test approach, we propose a novel test statistic via directional regression to determine the structural dimension in this paper.

Inference on the Primary Parameter of Interest with the Aid of Dimension Reduction Estimation

Summary As high dimensional data become routinely available in applied sciences, sufficient dimension reduction has been widely employed and its research has received considerable attention. However, with the majority of sufficient dimension reduction methodology focusing on the dimension reduction step, complete analysis and inference after dimension reduction have yet to receive much attention. We couple the strategy of sufficient dimension reduction with a flexible semiparametric model. We concentrate on inference with respect to the primary variables of interest, and we employ sufficient dimension reduction to bring down the dimension of the regression effectively. Extensive simulations demonstrate the efficacy of the method proposed, and a real data analysis is presented for illustration.

Partial moment-based sufficient dimension reduction

In this paper, a partial moment-based sufficient dimension reduction methodology is developed. This extends and generalizes some existing methods and enables us to test predictor contributions. All tests in the proposed method are done with chi-squared distributions.

Response dimension reduction for the conditional mean in multivariate regression

Sufficient dimension reduction methodologies in regression have been developed in the past decade, focusing mostly on predictors. Here, we propose a methodology to reduce the dimension of the response vector in multivariate regression, without loss of information about the conditional mean. The asymptotic distributions of dimension test statistics are chi-squared distributions, and an estimate of the dimension reduction subspace is asymptotically efficient. Moreover, the proposed methodology enables us to test response effects for the conditional mean. Properties of the proposed method are studied via simulation.

Sufficient dimension reduction for the conditional mean with a categorical predictor in multivariate regression

Recent sufficient dimension reduction methodologies in multivariate regression do not have direct application to a categorical predictor. For this, we define the multivariate central partial mean subspace and propose two methodologies to estimate it. The first method uses the ordinary least squares. Chi-squared distributed statistics for dimension tests are constructed, and an estimate of the target subspace is consistent and efficient. Moreover, the effects of continuous predictors can be tested without assuming any model. The second method extends Iterative Hessian Transformation to this context. For dimension estimation, permutation tests are used. Simulated and real data examples for illustrating various properties of the proposed methods are presented.

A novel moment-based sufficient dimension reduction approach in multivariate regression

Recently, a moment-based sufficient dimension reduction methodology in multivariate regression, focusing on the first two moments, was introduced. We present in this article a novel approach of the earlier method in roughly the same context. This novel method possesses several desirable properties that the earlier method did not have such as dimension tests with chi-squared distributions, predictor effects test without assuming any model, and so on. Simulated and real data examples are presented for studying various properties of the proposed method and for a numerical comparison to the earlier method.

Optimal sufficient dimension reduction for the conditional mean in multivariate regression

The aim of this article is to develop optimal sufficient dimension reduction methodology for the conditional mean in multivariate regression. The context is roughly the same as that of a related method by Cook & Setodji (2003), but the new method has several advantages. It is asymptotically optimal in the sense described herein and its test statistic for dimension always has a chi-squared distribution asymptotically under the null hypothesis. Additionally, the optimal method allows tests of predictor effects. A comparison of the two methods is provided.