Effective Dimension Reduction Space Research Articles

Wavelet-based estimation in a semiparametric regression model

In this paper, we introduce a wavelet-based method for estimating the effective dimension reduction (EDR) space in the semiparametric regression model introduced by Li [Sliced inverse regression for dimension reduction, J. Amer. Statist. Assoc. 86 (1991) 316–327]. This method is obtained by using linear wavelet estimators of the density and regression functions that are involved in the covariance matrix of conditional expectation whose eigenvectors span the EDR space. Then, consistency of the proposed estimators is proved. A simulation study that allows one to evaluate the performance of the proposal with comparison to existing methods is presented.

Overlapping sliced inverse regression for dimension reduction

Sliced inverse regression (SIR) is a pioneer tool for supervised dimension reduction. It identifies the effective dimension reduction space, the subspace of significant factors with intrinsic lower dimensionality. In this paper, we propose to refine the SIR algorithm through an overlapping slicing scheme. The new algorithm, called overlapping SIR (OSIR), is able to estimate the effective dimension reduction space and determine the number of effective factors more accurately. We show that such overlapping procedure has the potential to identify the information contained in the derivatives of the inverse regression curve, which helps to explain the superiority of OSIR. We also prove that OSIR algorithm is [Formula: see text]-consistent and verify its effectiveness by simulations and real applications.

Slice inverse regression with score functions

We consider non-linear regression problems where we assume that the response depends non-linearly on a linear projection of the covariates. We propose score function extensions to sliced inverse regression problems, both for the first-order and second-order score functions. We show that they provably improve estimation in the population case over the non-sliced versions and we study finite sample estimators and their consistency given the exact score functions. We also propose to learn the score function as well, in two steps, i.e., first learning the score function and then learning the effective dimension reduction space, or directly, by solving a convex optimization problem regularized by the nuclear norm. We illustrate our results on a series of experiments.

Variable importance assessment in sliced inverse regression for variable selection

ABSTRACTWe are interested in treating the relationship between a dependent variable y and a multivariate covariate in a semiparametric regression model. Since the purpose of most social, biological, or environmental science research is the explanation, the determination of the importance of the variables is a major concern. It is a way to determine which variables are the most important when predicting y. Sliced inverse regression methods allows to reduce the space of the covariate x by estimating the directions β that form an effective dimension reduction (EDR) space. The aim of this article is to propose a computational method based on importance variable measure (only relying on the EDR space) in order to select the most useful variables. The numerical behavior of this new method, implemented in R, is studied on a simulation study. An illustration on a real data is also provided.

Dimension reduction in functional regression with categorical predictor

In the present paper, we consider dimension reduction methods for functional regression with a scalar response and the predictors including a random curve and a categorical random variable. To deal with the categorical random variable, we propose three potential dimension reduction methods: partial functional sliced inverse regression, marginal functional sliced inverse regression and conditional functional sliced inverse regression. Furthermore, we investigate the relationships among the three methods. In addition, a new modified BIC criterion for determining the dimension of the effective dimension reduction space is developed. Real and simulation data examples are then presented to show the effectiveness of the proposed methods.

Probability-enhanced effective dimension reduction for classifying sparse functional data

We consider the classification of sparse functional data that are often encountered in longitudinal studies and other scientific experiments. To utilize the information from not only the functional trajectories but also the observed class labels, we propose a probability-enhanced method achieved by weighted support vector machine based on its Fisher consistency property to estimate the effective dimension reduction space. Since only a few measurements are available for some, even all, individuals, a cumulative slicing approach is suggested to borrow information across individuals. We provide justification for validity of the probability-based effective dimension reduction space, and a straightforward implementation that yields a low-dimensional projection space ready for applying standard classifiers. The empirical performance is illustrated through simulated and real examples, particularly in contrast to classification results based on the prominent functional principal component analysis.

Effective dimension reduction for sparse functional data.

We propose a method of effective dimension reduction for functional data, emphasizing the sparse design where one observes only a few noisy and irregular measurements for some or all of the subjects. The proposed method borrows strength across the entire sample and provides a way to characterize the effective dimension reduction space, via functional cumulative slicing. Our theoretical study reveals a bias-variance trade-off associated with the regularizing truncation and decaying structures of the predictor process and the effective dimension reduction space. A simulation study and an application illustrate the superior finite-sample performance of the method.

A new sliced inverse regression method for multivariate response

A semiparametric regression model of a q-dimensional multivariate response y on a p-dimensional covariate x is considered. A new approach is proposed based on sliced inverse regression (SIR) for estimating the effective dimension reduction (EDR) space without requiring a prespecified parametric model. The convergence at rate n of the estimated EDR space is shown. The choice of the dimension of the EDR space is discussed. Moreover, a way to cluster components of y related to the same EDR space is provided. Thus, the proposed multivariate SIR method can be used properly on each cluster instead of blindly applying it on all components of y. The numerical performances of multivariate SIR are illustrated on a simulation study. An application to the Minneapolis elementary schools data is also provided. Although the proposed methodology relies on SIR, it opens the door for new regression approaches with a multivariate response. They could be built similarly based on other reduction dimension methods.

Functional contour regression

In this paper, we propose functional contour regression (FCR) for dimension reduction in the functional regression context. FCR achieves dimension reduction using the empirical directions on the functional predictor in contours defined on the response variable. It is more efficient than the functional variants of the sliced inverse regression (SIR) method by exploiting inter-slice information. A modified BIC is used to determine the dimensionality of the effective dimension reduction space. We prove that FCR is consistent in estimating the functional regression parameters, and simulations show that the estimates given by our FCR method provide better prediction accuracy than other existing methods such as functional sliced inverse regression, functional inverse regression and wavelet SIR. The merit of FCR is further demonstrated by two real data examples.

A graphical tool for selecting the number of slices and the dimension of the model in SIR and SAVE approaches

Sliced inverse regression (SIR) and related methods were introduced in order to reduce the dimensionality of regression problems. In general semiparametric regression framework, these methods determine linear combinations of a set of explanatory variables X related to the response variable Y, without losing information on the conditional distribution of Y given X. They are based on a “slicing step” in the population and sample versions. They are sensitive to the choice of the number H of slices, and this is particularly true for SIR-II and SAVE methods. At the moment there are no theoretical results nor practical techniques which allows the user to choose an appropriate number of slices. In this paper, we propose an approach based on the quality of the estimation of the effective dimension reduction (EDR) space: the square trace correlation between the true EDR space and its estimate can be used as goodness of estimation. We introduce a naive bootstrap estimation of the square trace correlation criterion to allow selection of an “optimal” number of slices. Moreover, this criterion can also simultaneously select the corresponding suitable dimension K (number of the linear combination of X). From a practical point of view, the choice of these two parameters H and K is essential. We propose a 3D-graphical tool, implemented in R, which can be useful to select the suitable couple (H, K). An R package named “edrGraphicalTools” has been developed. In this article, we focus on the SIR-I, SIR-II and SAVE methods. Moreover the proposed criterion can be use to determine which method seems to be efficient to recover the EDR space, that is the structure between Y and X. We indicate how the proposed criterion can be used in practice. A simulation study is performed to illustrate the behavior of this approach and the need for selecting properly the number H of slices and the dimension K. A short real-data example is also provided.

Deciding the dimension of effective dimension reduction space for functional and high-dimensional data

In this paper, we consider regression models with a Hilbert-space-valued predictor and a scalar response, where the response depends on the predictor only through a finite number of projections. The linear subspace spanned by these projections is called the effective dimension reduction (EDR) space. To determine the dimensionality of the EDR space, we focus on the leading principal component scores of the predictor, and propose two sequential χ2 testing procedures under the assumption that the predictor has an elliptically contoured distribution. We further extend these procedures and introduce a test that simultaneously takes into account a large number of principal component scores. The proposed procedures are supported by theory, validated by simulation studies, and illustrated by a real-data example. Our methods and theory are applicable to functional data and high-dimensional multivariate data.

Application of the Bootstrap Approach to the Choice of Dimension and the α Parameter in the SIRα Method

To reduce the dimensionality of regression problems, sliced inverse regression approaches make it possible to determine linear combinations of a set of explanatory variables X related to the response variable Y in general semiparametric regression context. From a practical point of view, the determination of a suitable dimension (number of the linear combination of X) is important. In the literature, statistical tests based on the nullity of some eigenvalues have been proposed. Another approach is to consider the quality of the estimation of the effective dimension reduction (EDR) space. The square trace correlation between the true EDR space and its estimate can be used as goodness of estimation. In this article, we focus on the SIRα method and propose a naïve bootstrap estimation of the square trace correlation criterion. Moreover, this criterion could also select the α parameter in the SIRα method. We indicate how it can be used in practice. A simulation study is performed to illustrate the behavior of this approach.

Local likelihood regression in generalized linear single-index models with applications to microarray data

Searching for an effective dimension reduction space is an important problem in regression, especially for high-dimensional data such as microarray data. A major characteristic of microarray data consists in the small number of observations n and a very large number of genes p . This “large p, small n” paradigm makes the discriminant analysis for classification difficult. In order to offset this dimensionality problem a solution consists in reducing the dimension. Supervised classification is understood as a regression problem with a small number of observations and a large number of covariates. A new approach for dimension reduction is proposed. This is based on a semi-parametric approach which uses local likelihood estimates for single-index generalized linear models. The asymptotic properties of this procedure are considered and its asymptotic performances are illustrated by simulations. Applications of this method when applied to binary and multiclass classification of the three real data sets Colon, Leukemia and SRBCT are presented.

Principal Hessian Directions for regression with measurement error

SUMMARY We consider a nonlinear regression problem with predictors with measurement error. We assume that the response is related to unknown linear combinations of a p-dimensional predictor vector through an unknown link function. Instead of observing the predictors, we observe a surrogate vector with the property that its expectation is linearly related to the predictor vector with constant variance. We use an important linear transformation of the surrogates. Based on the transformed variables, we develop the modified Principal Hessian Directions method for estimating the subspace of the effective dimension reduction space. We derive the asymptotic variances of the modified Principal Hessian Directions estimators. Several examples are reported and comparisons are made with the sliced inverse regression method of Carroll & Li (1992).

An Adaptive Estimation of Dimension Reduction Space

SummarySearching for an effective dimension reduction space is an important problem in regression, especially for high dimensional data. We propose an adaptive approach based on semiparametric models, which we call the (conditional) minimum average variance estimation (MAVE) method, within quite a general setting. The MAVE method has the following advantages. Most existing methods must undersmooth the nonparametric link function estimator to achieve a faster rate of consistency for the estimator of the parameters (than for that of the nonparametric function). In contrast, a faster consistency rate can be achieved by the MAVE method even without undersmoothing the nonparametric link function estimator. The MAVE method is applicable to a wide range of models, with fewer restrictions on the distribution of the covariates, to the extent that even time series can be included. Because of the faster rate of consistency for the parameter estimators, it is possible for us to estimate the dimension of the space consistently. The relationship of the MAVE method with other methods is also investigated. In particular, a simple outer product gradient estimator is proposed as an initial estimator. In addition to theoretical results, we demonstrate the efficacy of the MAVE method for high dimensional data sets through simulation. Two real data sets are analysed by using the MAVE approach.

Nearest neighbor inverse regression

Sliced inverse regression (SIR), formally introduced by Li, is a very general procedure for performing dimension reduction in nonparametric regression. This paper considers a version of SIR in which the “slices” are determined by nearest neighbors and the response variable takes value possibly in a multidimensional space. It is shown, under general conditions, that the “effective dimension reduction space” can be estimated with rate $n^{-1/2}$ where $n$ is the sample size.

Determining the Dimension in Sliced Inverse Regression and Related Methods

Sliced inverse regression (SIR) and principal Hessian directions aim to reduce the dimensionality of regression problems. An important step in the method is the determination of a suitable dimension. Although statistical tests based on the nullity eigenvalues are usually suggested, this article focuses on the quality of the estimation of the effective dimension reduction (EDR) spaces. Essentially, the goal is to retain only sufficiently stable subspaces. The goodness of the estimation is measured by the squared trace correlation between the subspaces of the EDR space and their estimates. Asymptotic expansions are derived and estimates deduced. Simulations give an insight on the behavior of the criterion and indicate how it can be used in practice.