Independence Screening Procedure Research Articles

Feature screening for ultra-high-dimensional data via multiscale graph correlation

Variable screening plays a crucial role in ultra-high-dimensional data analysis. In this article, we establish a sure independence screening procedure based on the multiscale graph correlation (MGC), a new frame to generalize distance correlation, which can identify the monotonous or non monotonic relationship between predictors and response, and also enjoy the same sure screening property as the DC-SIS. Besides, we extend the method to right-censored survival data in two ways, the Kaplan-Meier estimator and composite quantile, respectively, and build the corresponding sure screening properties. Through numerical simulation, the results show that MGC-based screening methods have better performance than other methods when complicated non linear relationships exist for both complete data and right-censored data. Furthermore, we apply the proposed methods to two real datasets to examine the ranking ability and model prediction accuracy.

Support vector machine in ultrahigh-dimensional feature space

Classification and feature selection play an important role in knowledge discovery in high-dimensional data. Although penalized Support Vector Machine (SVM) is among the most powerful methods for classification and automatic feature selection in high-dimensional feature space, it is not directly applicable to ultrahigh-dimensional cases, wherein the number of features far exceeds the sample size. In this paper, we suggest an efficient two-step method for simultaneous classification and identifying important features in the setting of ultrahigh-dimensional models. Specifically, we first develop an independence screening procedure to reduce the dimensionality of the feature space to a moderate scale, and then penalized support vector machine is applied to the dimension-reduced feature space to select important features further and estimate the coefficients, via a (penalized) model fit. Implementation of the suggested two-step method is not limited by the dimensionality of the models and entails much less computational cost. Numerical examples and a real data analysis are used to demonstrate the finite sample performance of our proposal.

Feature selection in ultrahigh-dimensional additive models with heterogeneous frequency component functions

In this paper, we consider feature selection in ultrahigh-dimensional additive models with heterogeneous frequency component functions. Firstly, we introduce a new concept, weighted sum of squared conditional correlations (WSSCC), which measures the correlation between a random variable and its function. Afterwards, we propose a sure independence screening procedure based on WSSCC (WSSCCSIS), whose sure screening property is established. Furthermore, a sequential feature selection procedure based on WSSCC (WSSCCFR) is proposed. Numerical studies including comprehensive simulation and a real data analysis are carried out to demonstrate the advantage of our method over other existing approaches.

Group Feature Screening Based on Information Gain Ratio for Ultrahigh-Dimensional Data

Most model-free feature screening approaches focus on the -individual predictor; therefore, they are not able to incorporate structured predictors like grouped variables. In this article, we propose a group screening procedure via the information gain ratio for a classification model, which is a direct extension of the original sure independence screening procedure and also model-free. The proposed method yields a better screening performance and classification accuracy. It is demonstrated that the proposed group screening method possesses the sure screening property and ranking consistency properties under certain regularity conditions. Through simulation studies and real-world data analysis, we demonstrate the proposed method with the finite sample performance.

A Combined Feature Screening Approach of Random Forest and Filterbased Methods for Ultra-high Dimensional Data

Background: Various feature (variable) screening approaches have been proposed in the past decade to mitigate the impact of ultra-high dimensionality in classification and regression problems, including filter based methods such as sure independence screening, and wrapper based methods such as random forest. However, the former type of methods rely heavily on strong modelling assumptions while the latter ones requires an adequate sample size to make the data speak for themselves. These requirements can seldom be met in biochemical studies in cases where we have only access to ultra-high dimensional data with a complex structure and a small number of observations. Objective: In this research, we want to investigate the possibility of combining both filter based screening methods and random forest based screening methods in the regression context. Method: We have combined four state-of-art filter approaches, namely, sure independence screening (SIS), robust rank correlation based screening (RRCS), high dimensional ordinary least squares projection (HOLP) and a model free sure independence screening procedure based on the distance correlation (DCSIS) from the statistical community with a random forest based Boruta screening method from the machine learning community for regression problems. Result: Among all the combined methods, RF-DCSIS performs better than the other methods in terms of screening accuracy and prediction capability on the simulated scenarios and real benchmark datasets. Conclusion: By empirical study from both extensive simulation and real data, we have shown that both filter based screening and random forest based screening have their pros and cons, while a combination of both may lead to a better feature screening result and prediction capability.

Feature Screening for High-Dimensional Survival Data via Censored Quantile Correlation

This paper proposes a new sure independence screening procedure for high-dimensional survival data based on censored quantile correlation (CQC). This framework has two distinctive features: 1) Via incorporating a weighting scheme, our metric is a natural extension of quantile correlation (QC), considered by Li (2015), to handle high-dimensional survival data; 2) The proposed method not only is robust against outliers, but also can discover the nonlinear relationship between independent variables and censored dependent variable. Additionally, the proposed method enjoys the sure screening property under certain technical conditions. Simulation results demonstrate that the proposed method performs competitively on survival datasets of high-dimensional predictors.

Projection correlation between scalar and vector variables and its use in feature screening with multi-response data

In this article, we introduce a new methodology to perform feature screening for ultrahigh dimensional data with multivariate responses. Several extant screening procedures are available for multivariate responses, but they may be adversely affected by heavy-tailed observations or the dimension of multivariate responses. In order to attack these challenges, we first introduce a nonparametric coefficient, called projection correlation, to measure the departure of dependence between a scalar variable X and a vector variable . It takes values between zero and one, does not require any moment conditions on X and , and is zero if and only if X and are independent. Based on its estimation that has desirable theoretical properties, such as algebraic simplicity and consistency, we present a novel sure independence screening procedure, which enjoys the desirable sure screening property. Numerical results demonstrate the effectiveness of the proposed procedure in comparison with the existing counterparts.

Ultra-high dimensional variable screening via Gram–Schmidt orthogonalization

Independence screening procedure plays a vital role in variable selection when the number of variables is massive. However, high dimensionality of the data may bring in many challenges, such as multicollinearity or high correlation (possibly spurious) between the covariates, which results in marginal correlation being unreliable as a measure of association between the covariates and the response. We propose a novel and simple screening procedure called Gram–Schmidt screening (GSS) by integrating the classical Gram–Schmidt orthogonalization and the sure independence screening technique, which takes into account high correlations between the covariates in a data-driven way. GSS could successfully discriminate between the relevant and the irrelevant variables to achieve a high true positive rate without including many irrelevant and redundant variables, which offers a new perspective for screening method when the covariates are highly correlated. The practical performance of GSS was shown by comparative simulation studies and analysis of two real datasets.

A consistent variable screening procedure with family-wise error control

ABSTRACTVariable selection in ultra-high-dimensional data analysis is of critical importance in diverse scientific fields. There are many methods proposed for this problem in the past few decades. However, no attention has been paid on the model size produced by screening procedures and the difficulty of determining the screening threshold for practical use, which is key to real applications. Specifically, the pre-defined model size in a screening procedure lacks theoretical support, and users have no idea about the true positives and false positives in the final results. To solve this problem, we developed a new consistent independence screening procedure called HHG-CIS. The screening threshold of HHG-CIS can be set to explicitly control the family-wise error rate, thus it is easy to use in practice. Furthermore, HHG-CIS is model-free, robust to outliers, and outperforms other methods in models with interactions. We showed these advantages of HHG-CIS through extensive simulations and a real data application.

High‐dimensional variable screening under multicollinearity

Variable screening is of fundamental importance in linear regression models when the number of predictors far exceeds the number of observations. Multicollinearity is a common phenomenon in high‐dimensional settings, in which two or more predictor variables are highly correlated, leading to the notorious difficulty for high‐dimensional variable screening. Sure independence screening (SIS) procedure can greatly reduce the dimensionality, but it may break down when the predictors are highly correlated. By combing the factor modelling with SIS, the profiled independence screening (PIS) approach was proposed. However, under a spiked population model, the profiled predictors could not be guaranteed to be uncorrelated and PIS may therefore be misleading. Instead of assuming either the predictors are uncorrelated as in SIS or the profiled predictors are uncorrelated as in PIS, a more general and challenging scenario is considered in which the predictors can be highly correlated. A so‐called preconditioned PIS (PPIS) method is proposed that produces asymptotically uncorrelated profiled predictors and thus leads to consistent model selection results under a spiked population model. Compared with PIS, the proposed method could handle the complex multicollinearity case, such as a spiked population model with a slow spectrum decay of population covariance matrix, while keeping the calculation simple. The promising performance of the proposed PPIS method will be illustrated via extensive simulation studies and two real examples.

Group feature screening via the F statistic

Feature screening is crucial in the analysis of ultrahigh dimensional data, where the number of variables (features) is in an exponential order of the number of observations. In various ultrahigh dimensional data, variables are naturally grouped, giving us a good rationale to develop a screening method using joint effect of multiple variables. In this article, we propose a group screening procedure via the F-test statistic. The proposed method is a direct extension of the original sure independence screening procedure, when the group information is known, for example, from prior knowledge. Under certain regularity conditions, we prove that the proposed group screening procedure possesses the sure screening property that selects all effective groups with a probability approaching one at an exponential rate. We use simulations to demonstrate the advantages of the proposed method and show its application in a genome-wide association study. We conclude that the grouping method is very useful in the analysis of ultrahigh dimensional data, as the optimal F-test can detect true signals with desired properties.

Variable screening with multiple studies

Advancement in technology has generated abundant high-dimensional data that allows integration of multiple relevant studies. Due to their huge computational advantage, variable screening methods based on marginal correlation have become promising alternatives to the popular regularization methods for variable selection. However, all these screening methods are limited to single study so far. In this paper, we consider a general framework for variable screening with multiple related studies, and further propose a novel two-step screening procedure using a self-normalized estimator for high-dimensional regression analysis in this framework. Compared to the one-step procedure and rank-based sure independence screening (SIS) procedure, our procedure greatly reduces false negative errors while keeping a low false positive rate. Theoretically, we show that our procedure possesses the sure screening property with weaker assumptions on signal strengths and allows the number of features to grow at an exponential rate of the sample size. In addition, we relax the commonly used normality assumption and allow sub-Gaussian distributions. Simulations and a real transcriptomic application illustrate the advantage of our method as compared to the rank-based SIS method.

A sure independence screening procedure for ultra-high dimensional partially linear additive models

ABSTRACTWe introduce a two-step procedure, in the context of ultra-high dimensional additive models, which aims to reduce the size of covariates vector and distinguish linear and nonlinear effects among nonzero components. Our proposed screening procedure, in the first step, is constructed based on the concept of cumulative distribution function and conditional expectation of response in the framework of marginal correlation. B-splines and empirical distribution functions are used to estimate the two above measures. The sure screening property of this procedure is also established. In the second step, a double penalization based procedure is applied to identify nonzero and linear components, simultaneously. The performance of the designed method is examined by several test functions to show its capabilities against competitor methods when the distribution of errors is varied. Simulation studies imply that the proposed screening procedure can be applied to the ultra-high dimensional data and well detect the influential covariates. It also demonstrate the superiority in comparison with the existing methods. This method is also applied to identify most influential genes for overexpression of a G protein-coupled receptor in mice.

Joint screening of ultrahigh dimensional variables for family-based genetic studies

BackgroundMixed models are a useful tool for evaluating the association between an outcome variable and genetic variables from a family-based genetic study, taking into account the kinship coefficients. When there are ultrahigh dimensional genetic variables (ie, p ≫ n), it is challenging to fit any mixed effect model.MethodsWe propose a two-stage strategy, screening genetic variables in the first stage and then fitting the mixed effect model in the second stage to those variables that survive the screening. For the screening stage, we can use the sure independence screening (SIS) procedure, which fits the mixed effect model to one genetic variable at a time. Because the SIS procedure may fail to identify those marginally unimportant but jointly important genetic variables, we propose a joint screening (JS) procedure that screens all the genetic variables simultaneously. We evaluate the performance of the proposed JS procedure via a simulation study and an application to the GAW20 data.ResultsWe perform the proposed JS procedure on the GAW20 representative simulated data set (n = 680 participant(s) and p = 463,995 CpG cytosine-phosphate-guanine [CpG] sites) and select the top d = ⌊n/ log(n)⌋ variables. Then we fit the mixed model using these top variables. Under significance level, 5%, 43 CpG sites are found to be significant. Some diagnostic analyses based on the residuals show the fitted mixed model is appropriate.ConclusionsAlthough the GAW20 data set is ultrahigh dimensional and family-based having within group variances, we were successful in performing subset selection using a two-step strategy that is computationally simple and easy to understand.

A ten-long non-coding RNA signature for predicting prognosis of patients with cervical cancer.

PurposeThe aim of the present study was to construct a novel long non-coding RNA (lncRNA) signature to predict the prognosis of patients with cervical cancer (CC).Materials and methodsWe downloaded lncRNA expression profiles and clinical characteristics from The Cancer Genome Atlas database and randomly divided them into a training dataset (n=200) and a testing dataset (n=87). Using a Cox-based iterative sure independence screening procedure combined with a resampling technique, a lncRNA signature was calculated from prognostic lncRNAs in the training dataset and was independently verified in the testing and the entire datasets. In addition, multivariate Cox regression and further stratified analyses were performed, taking into consideration the lncRNA signature as well as other clinical characteristics. Finally, we predicted the underlying functional effects of the prognostic lncRNAs by using Gene Ontology and Kyoto Encyclopedia of Genes and Genomes pathway enrichment analyses.ResultsWe constructed a promising ten-lncRNA signature that was significantly associated with the prognosis of CC on the basis of a risk score formula. The risk score was used to classify patients into high-risk and low-risk groups with different overall survival in the training dataset, and was confirmed in the testing and entire datasets. Compared with the clinical factors, the ten-lncRNA signature was found to be an independent prognostic indicator and displayed robust prognostic performance. A functional analysis indicated that these ten lncRNAs were enriched in immune response, cell adhesion molecules and nuclear factor kappa B signaling.ConclusionOur results demonstrated that this ten-lncRNA signature may serve as a prognostic biomarker for patients with CC.

Variable selection and structure identification for ultrahigh-dimensional partially linear additive models with application to cardiomyopathy microarray data

In this paper, we introduce a two-step procedure, in the context of ultrahigh-dimensional additive models, to identify nonzero and linear components. We first develop a sure independence screening procedure based on the distance correlation between predictors and marginal distribution function of the response variable to reduce the dimensionality of the feature space to a moderate scale. Then a double penalization based procedure is applied to identify nonzero and linear components, simultaneously. We conduct extensive simulation experiments to evaluate the numerical performance of the proposed method and analyze a cardiomyopathy microarray data for an illustration. Numerical studies confirm the fine performance of the proposed method for various semiparametric models.

A Generic Sure Independence Screening Procedure

ABSTRACTExtracting important features from ultra-high dimensional data is one of the primary tasks in statistical learning, information theory, precision medicine, and biological discovery. Many of the sure independent screening methods developed to meet these needs are suitable for special models under some assumptions. With the availability of more data types and possible models, a model-free generic screening procedure with fewer and less restrictive assumptions is desirable. In this article, we propose a generic nonparametric sure independence screening procedure, called BCor-SIS, on the basis of a recently developed universal dependence measure: Ball correlation. We show that the proposed procedure has strong screening consistency even when the dimensionality is an exponential order of the sample size without imposing sub-exponential moment assumptions on the data. We investigate the flexibility of this procedure by considering three commonly encountered challenging settings in biological discovery or precision medicine: iterative BCor-SIS, interaction pursuit, and survival outcomes. We use simulation studies and real data analyses to illustrate the versatility and practicability of our BCor-SIS method. Supplementary materials for this article are available online.

A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates

In this paper, a new robust and efficient estimation approach based on local modal regression is proposed for partially linear models with large-dimensional covariates. We show that the resulting estimators for both parametric and nonparametric components are more efficient in the presence of outliers or heavy-tail error distribution, and as asymptotically efficient as the corresponding least squares estimators when there are no outliers and the error distribution is normal. We also establish the asymptotic properties of proposed estimators when the covariate dimension diverges at the rate of $$o\left( {\sqrt{n} } \right) \mathrm{{ }}$$ . To achieve sparsity and enhance interpretability, we develop a variable selection procedure based on SCAD penalty to select significant parametric covariates and show that the method enjoys the oracle property under mild regularity conditions. Moreover, we propose a practical modified MEM algorithm for the proposed procedures. Some Monte Carlo simulations and a real data are conducted to illustrate the finite sample performance of the proposed estimators. Finally, based on the idea of sure independence screening procedure proposed by Fan and Lv (J R Stat Soc 70:849–911, 2008), a robust two-step approach is introduced to deal with ultra-high dimensional data.

FGWAS: Functional genome wide association analysis

Functional phenotypes (e.g., subcortical surface representation), which commonly arise in imaging genetic studies, have been used to detect putative genes for complexly inherited neuropsychiatric and neurodegenerative disorders. However, existing statistical methods largely ignore the functional features (e.g., functional smoothness and correlation). The aim of this paper is to develop a functional genome-wide association analysis (FGWAS) framework to efficiently carry out whole-genome analyses of functional phenotypes. FGWAS consists of three components: a multivariate varying coefficient model, a global sure independence screening procedure, and a test procedure. Compared with the standard multivariate regression model, the multivariate varying coefficient model explicitly models the functional features of functional phenotypes through the integration of smooth coefficient functions and functional principal component analysis. Statistically, compared with existing methods for genome-wide association studies (GWAS), FGWAS can substantially boost the detection power for discovering important genetic variants influencing brain structure and function. Simulation studies show that FGWAS outperforms existing GWAS methods for searching sparse signals in an extremely large search space, while controlling for the family-wise error rate. We have successfully applied FGWAS to large-scale analysis of data from the Alzheimer's Disease Neuroimaging Initiative for 708 subjects, 30,000 vertices on the left and right hippocampal surfaces, and 501,584 SNPs.

A robust variable screening method for high-dimensional data

ABSTRACTIn practice, the presence of influential observations may lead to misleading results in variable screening problems. We, therefore, propose a robust variable screening procedure for high-dimensional data analysis in this paper. Our method consists of two steps. The first step is to define a new high-dimensional influence measure and propose a novel influence diagnostic procedure to remove those unusual observations. The second step is to utilize the sure independence screening procedure based on distance correlation to select important variables in high-dimensional regression analysis. The new influence measure and diagnostic procedure that we developed are model free. To confirm the effectiveness of the proposed method, we conduct simulation studies and a real-life data analysis to illustrate the merits of the proposed approach over some competing methods. Both the simulation results and the real-life data analysis demonstrate that the proposed method can greatly control the adverse effect after detecting and removing those unusual observations, and performs better than the competing methods.