Dimension Reduction In Regression Research Articles

Data-driven slicing for dimension reduction in regressions: A likelihood-ratio approach

Data-driven slicing for dimension reduction in regressions: A likelihood-ratio approach

It's all relative: Regression analysis with compositional predictors.

Compositional data reside in a simplex and measure fractions or proportions of parts to a whole. Most existing regression methods for such data rely on log-ratio transformations that are inadequate or inappropriate in modeling high-dimensional data with excessive zeros and hierarchical structures. Moreover, such models usually lack a straightforward interpretation due to the interrelation between parts of a composition. We develop a novel relative-shift regression framework that directly uses proportions as predictors. The new framework provides a paradigm shift for regression analysis with compositional predictors and offers a superior interpretation of how shifting concentration between parts affects the response. New equi-sparsity and tree-guided regularization methods and an efficient smoothing proximal gradient algorithm are developed to facilitate feature aggregation and dimension reduction in regression. A unified finite-sample prediction error bound is derived for the proposed regularized estimators. We demonstrate the efficacy of the proposed methods in extensive simulation studies and a real gut microbiome study. Guided by the taxonomy of the microbiome data, the framework identifies important taxa at different taxonomic levels associated with the neurodevelopment of preterminfants.

An endorectal ultrasound-based radiomics signature for preoperative prediction of lymphovascular invasion of rectal cancer

ObjectiveTo investigate whether radiomics based on ultrasound images can predict lymphovascular invasion (LVI) of rectal cancer (RC) before surgery.MethodsA total of 203 patients with RC were enrolled retrospectively, and they were divided into a training set (143 patients) and a validation set (60 patients). We extracted the radiomic features from the largest gray ultrasound image of the RC lesion. The intraclass correlation coefficient (ICC) was applied to test the repeatability of the radiomic features. The least absolute shrinkage and selection operator (LASSO) was used to reduce the data dimension and select significant features. Logistic regression (LR) analysis was applied to establish the radiomics model. The receiver operating characteristic (ROC) curve, calibration curve, and decision curve analysis (DCA) were used to evaluate the comprehensive performance of the model.ResultsAmong the 203 patients, 33 (16.7%) were LVI positive and 170 (83.7%) were LVI negative. A total of 5350 (90.1%) radiomic features with ICC values of ≥ 0.75 were reported, which were subsequently subjected to hypothesis testing and LASSO regression dimension reduction analysis. Finally, 15 selected features were used to construct the radiomics model. The area under the curve (AUC) of the training set was 0.849, and the AUC of the validation set was 0.781. The calibration curve indicated that the radiomics model had good calibration, and DCA demonstrated that the model had clinical benefits.ConclusionThe proposed endorectal ultrasound-based radiomics model has the potential to predict LVI preoperatively in RC.

Sparse sliced inverse regression for high dimensional data analysis

BackgroundDimension reduction and variable selection play a critical role in the analysis of contemporary high-dimensional data. The semi-parametric multi-index model often serves as a reasonable model for analysis of such high-dimensional data. The sliced inverse regression (SIR) method, which can be formulated as a generalized eigenvalue decomposition problem, offers a model-free estimation approach for the indices in the semi-parametric multi-index model. Obtaining sparse estimates of the eigenvectors that constitute the basis matrix that is used to construct the indices is desirable to facilitate variable selection, which in turn facilitates interpretability and model parsimony.ResultsTo this end, we propose a group-Dantzig selector type formulation that induces row-sparsity to the sliced inverse regression dimension reduction vectors. Extensive simulation studies are carried out to assess the performance of the proposed method, and compare it with other state of the art methods in the literature.ConclusionThe proposed method is shown to yield competitive estimation, prediction, and variable selection performance. Three real data applications, including a metabolomics depression study, are presented to demonstrate the method’s effectiveness in practice.

Dimension reduction in binary response regression: A joint modeling approach

Categorical responses cause no conceptual complications for dimension reduction in regression, but the performance of some methods may suffer in this context and hence supervised dimension reduction in practice must recognize the nature of the response. Using a continuous latent variable to represent an unobserved response underlying the binary response, a joint model is proposed for dimension reduction in binary regression. The minimal sufficient linear reduction is obtained, and an efficient expectation maximization algorithm is developed for carrying out maximum likelihood estimation. Simulated examples and an application to a dataset concerning the identification of handwritten digits are presented to compare the performance of the proposed method with that of existing methods.

Dimensionality determination: A thresholding double ridge ratio approach

Underdetermination of model dimensionality (order) is a longstanding problem when existing eigendecomposition-based criteria are used. To alleviate this difficulty, we propose a thresholding double ridge ratio criterion in this paper. Unlike all existing eigendecomposition-based criteria, the proposed criterion can provide a consistent estimate even when there are several local minima. For illustration, we present the generic strategy with three important applications: dimension reduction in regressions with fixed and divergent dimensions; model checking with local alternative models; and ultra-high dimensional approximate factor models. Numerical studies are conducted to examine the finite sample performance of the proposed method and a real data example is analyzed for illustration.

Graph-Assisted Inverse Regression for Count Data and Its Application to Sequencing Data

Multivariate count data, such as sequencing reads in genomics, are often connected to a clinical phenotype of interest. We develop a flexible framework for dimension reduction in regression, with predictors that are correlated counts, by modeling the conditional distribution of the predictors, given the response, using a pairwise Poisson graphical model. This new framework, called network-based inverse regression for counts, allows us to derive a sufficient reduction of the predictors, while adjusting for the dependence structure among them. We propose a regularized criterion for estimating both the reduction structure and the network structure. The estimation algorithm can be implemented efficiently on a parallel computer. We also introduce an adaptive version and a sparse variant of the proposed procedure. The methods are evaluated on simulated data and are applied to a gut microbiome sequencing dataset. Supplementary materials for this article are available online.

Flexible dimension reduction in regression

Flexible dimension reduction in regression

A new estimator for efficient dimension reduction in regression

In this paper we propose a new estimator for efficient dimension reduction in regression, based on the work in Luo et al. (2014). Previous efficient estimators have been proposed by multiple authors, however under additional restrictive assumptions on the conditional variance of the response variable given the predictor vector. These assumptions also complicate the implementation. In contrast, the new estimator employs no such assumptions, and thus is far more applicable and more convenient to use. By an extended double-robustness property, it reaches asymptotic efficiency under fairly general conditions. Its finite-sample effectiveness is further illustrated by simulation studies and a real data example.

Diagnostic studies in sufficient dimension reduction

Sufficient dimension reduction in regression aims to reduce the predictor dimension by replacing the original predictors with some set of linear combinations of them without loss of information. Numerous dimension reduction methods have been developed based on this paradigm. However, little effort has been devoted to diagnostic studies within the context of dimension reduction. In this paper we introduce methods to check goodness-of-fit for a given dimension reduction subspace. The key idea is to extend the so-called distance correlation to measure the conditional dependence relationship between the covariates and the response given a reduction subspace. Our methods require minimal assumptions, which are usually much less restrictive than the conditions needed to justify the original methods. Asymptotic properties of the test statistic are studied. Numerical examples demonstrate the effectiveness of the proposed approach.

Estimation and inference in functional single-index models

We propose a functional single-index model (FSiM) to study the link between a scalar response variable and multiple functional predictors, in which the mean of the response is related to the linear predictors via an unknown link function. The FSiM serves as a good tool for dimension reduction in regression with multiple predictors and it is more flexible than functional linear models. Assuming that the functional predictors are observed at discrete points, we use B-spline basis functions to estimate the slope functions and the link function based on the least-squares criterion, and propose an iterative estimating procedure. Moreover, we provide uniform convergence rates of the proposed spline estimators in the FSiM, and construct asymptotic simultaneous confidence bands for the slope functions for inference. Our proposed method is illustrated by simulation studies and by an analysis of a diffusion tensor imaging data application.

Transformed sufficient dimension reduction

We propose a general framework for dimension reduction in regression to fill the gap between linear and fully nonlinear dimension reduction. The main idea is to first transform each of the raw predictors monotonically and then search for a low-dimensional projection in the space defined by the transformed variables. Both user-specified and data-driven transformations are suggested. In each case, the methodology is first discussed in generality and then a representative method is proposed and evaluated by simulation. The proposed methods are applied to a real dataset.

Cluster-based regularized sliced inverse regression for forecasting macroeconomic variables

This paper concerns the dimension reduction in regression for large data set. The authors introduce a new method based on the sliced inverse regression approach, called cluster-based regularized sliced inverse regression. The proposed method not only keeps the merit of considering both response and predictors’ information, but also enhances the capability of handling highly correlated variables. It is justified under certain linearity conditions. An empirical application on a macroeconomic data set shows that the proposed method has outperformed the dynamic factor model and other shrinkage methods.

Sufficient dimension reduction in multivariate regressions with categorical predictors

In this paper, we present a novel sufficient dimension reduction method for multivariate regressions with categorical predictors. We adopt ideas from a previous work byChiaromonte et al. (2002) who proposed sufficient dimension reduction in regressions with categorical predictors and the work by Li et al. (2008) who proposed the projective-resampling idea to multivariate response problems. In addition, we incorporate a variable selection procedure. Simulation studies show the efficacy of our method. We present a real data analysis through our proposed method to discover new association between personal characteristics and dietary factors which influence plasma beta-carotene and retinol levels in human serum.

Sufficient dimension reduction in regression with missing predictors

Existing sufficient dimension reduction methods often resort to the complete- case analysis when the predictors are subject to missingness. The complete-case analysis is inefficient even with the missing completely at random mechanism be- cause all incomplete cases are discarded. In this paper, we introduce a nonparamet- ric imputation procedure for semiparametric regressions with missing predictors. We establish the consistency of the nonparametric imputation under the missing at random mechanism that allows the missingness to depend exclusively upon the completely observed response. When the missingness depends on both the com- pletely observed predictors and the response, we propose a parametric method to impute the missing predictors. We demonstrate the estimation consistency of the parametric imputation method through several synthetic examples. Our proposals are illustrated through comprehensive simulations and a data application.

Dimension reduction transfer function model

The dimension reduction in regression is an efficient method of overcoming the curse of dimensionality in non-parametric regression. Motivated by recent developments for dimension reduction in time series, an empirical extension of central mean subspace in time series to a single-input transfer function model is performed in this paper. Here, we use central mean subspace as a tool of dimension reduction for bivariate time series in the case when the dimension and lag are known and estimate the central mean subspace through the Nadaraya–Watson kernel smoother. Furthermore, we develop a data-dependent approach based on a modified Schwarz Bayesian criterion to estimate the unknown dimension and lag. Finally, we show that the approach in bivariate time series works well using an expository demonstration, two simulations, and a real data analysis such as El Niño and fish Population.

Dimension Reduction in Regressions through Weighted Variance Estimation

Because sliced inverse regression (SIR) using the conditional mean of the inverse regression fails to recover the central subspace when the inverse regression mean degenerates, sliced average variance estimation (SAVE) using the conditional variance was proposed in the sufficient dimension reduction literature. However, the efficacy of SAVE depends heavily upon the number of slices. In the present article, we introduce a class of weighted variance estimation (WVE), which, similar to SAVE and simple contour regression (SCR), uses the conditional variance of the inverse regression to recover the central subspace. The strong consistency and the asymptotic normality of the kernel estimation of WVE are established under mild regularity conditions. Finite sample studies are carried out for comparison with existing methods and an application to a real data is presented for illustration.

Sliced inverse moment regression using weighted chi-squared tests for dimension reduction

We propose a new method for dimension reduction in regression using the first two inverse moments. We develop corresponding weighted chi-squared tests for the dimension of the regression. The proposed method considers linear combinations of sliced inverse regression (SIR) and the method using a new candidate matrix which is designed to recover the entire inverse second moment subspace. The optimal combination may be selected based on the p-values derived from the dimension tests. Theoretically, the proposed method, as well as sliced average variance estimate (SAVE), is more capable of recovering the complete central dimension reduction subspace than SIR and principle Hessian directions (pHd). Therefore it can substitute for SIR, pHd, SAVE, or any linear combination of them at a theoretical level. Simulation study indicates that the proposed method may have consistently greater power than SIR, pHd, and SAVE.

On dimension folding of matrix- or array-valued statistical objects

We consider dimension reduction for regression or classification in which the predictors are matrix- or array-valued. This type of predictor arises when measurements are obtained for each combination of two or more underlying variables--for example, the voltage measured at different channels and times in electroencephalography data. For these applications, it is desirable to preserve the array structure of the reduced predictor (e.g., time versus channel), but this cannot be achieved within the conventional dimension reduction formulation. In this paper, we introduce a dimension reduction method, to be called dimension folding, for matrix- and array-valued predictors that preserves the array structure. In an application of dimension folding to an electroencephalography data set, we correctly classify 97 out of 122 subjects as alcoholic or nonalcoholic based on their electroencephalography in a cross-validation sample.

Sufficient dimension reduction in regressions through cumulative Hessian directions

To reduce the predictors dimension without loss of information on the regression, we develop in this paper a sufficient dimension reduction method which we term cumulative Hessian directions. Unlike many other existing sufficient dimension reduction methods, the estimation of our proposal avoids completely selecting the tuning parameters such as the number of slices in slicing estimation or the bandwidth in kernel smoothing. We also investigate the asymptotic properties of our proposal when the predictors dimension diverges. Illustrations through simulations and an application are presented to evidence the efficacy of our proposal and to compare it with existing methods.