Independence Screening Research Articles

On correlation rank screening for ultra-high dimensional competing risks data

In recent years, numerous feature screening schemes have been developed for ultra-high dimensional standard survival data with only one failure event. Nevertheless, existing literature pays little attention to related investigations for competing risks data, in which subjects suffer from multiple mutually exclusive failures. In this article, we develop a new marginal feature screening for ultra-high dimensional time-to-event data to allow for competing risks. The proposed procedure is model-free, and robust against heavy-tailed distributions and potential outliers for time to the type of failure of interest. Apart from this, it is invariant to any monotone transformation of event time of interest. Under rather mild assumptions, it is shown that the newly suggested approach possesses the ranking consistency and sure independence screening properties. Some numerical studies are conducted to evaluate the finite-sample performance of our method and make a comparison with its competitor, while an application to a real data set is provided to serve as an illustration.

Covariate Information Number for Feature Screening in Ultrahigh-Dimensional Supervised Problems

Contemporary high-throughput experimental and surveying techniques give rise to ultrahigh-dimensional supervised problems with sparse signals; that is, a limited number of observations (n), each with a very large number of covariates , only a small share of which is truly associated with the response. In these settings, major concerns on computational burden, algorithmic stability, and statistical accuracy call for substantially reducing the feature space by eliminating redundant covariates before the use of any sophisticated statistical analysis. Along the lines of Pearson’s correlation coefficient-based sure independence screening and other model- and correlation-based feature screening methods, we propose a model-free procedure called covariate information number-sure independence screening (CIS). CIS uses a marginal utility connected to the notion of the traditional Fisher information, possesses the sure screening property, and is applicable to any type of response (features) with continuous features (response). Simulations and an application to transcriptomic data on rats reveal the comparative strengths of CIS over some popular feature screening methods. Supplementary materials for this article are available online.

Wall Density-Controlled Thermal Conductive and Mechanical Properties of Three-Dimensional Vertically Aligned Boron Nitride Network-Based Polymeric Composites

Polymeric composites with good thermal conductive and improved mechanical properties are in high demand in the thermal management materials. Construction of a three-dimensional (3D) structure has been proved to be an effective method to obtain polymeric composites with improved through-plane thermal conductivity (TC) for efficient thermal management of electronics. However, the TC enhancement of the obtained polymeric composites is limited, mainly due to poor control of the 3D thermal conductive network. Additionally, achieving high thermal conductive properties and enhanced mechanical properties simultaneously is of great challenge for polymeric composites. In this work, a 3D boron nitride framework (BNF) with a well-defined vertically aligned open structure and designed wall density fabricated by a unidirectional freezing technique was applied. The as-prepared BNF/polyethylene glycol (PBNF) composites exhibit enhanced through-plane TC, excellent thermal transfer capability (ΔTmax = 34 °C), and improved mechanical properties (Young's modulus enhancement up to 356%) simultaneously, making it attractive to thermal management applications. Strong correlation between the TC and mechanical properties of the PBNF composites and the wall density of the BNF scaffolds was found, providing opportunities to tune the TC and mechanical properties through the controlling of wall density. Furthermore, the models between TC and Young's modulus of PBNF composites were established by using the data-driven method "sure independence screening and sparsifying operator", which enables us to predict TC and Young's modulus of the polymeric composites for designing promising composite materials. The design principles and fabrication strategies proposed in this work could be important for developing advanced composite materials.

Descriptors of atoms and structure information for predicting properties of crystalline materials

Machine learning (ML) has increasingly been of interest in the design of new materials. However, it is still challenging to exploit an ML model in this field because its performance highly depends on the representation of materials, its properties, and the amount of data. In this study, for the cases of prediction of properties of crystalline materials, we explore a systematic comparison of two state-of-the-art frameworks: Crystal Graph Convolutional Neural Networks (CGCNNs) and the Sure Independence Screening and Sparsifying Operator (SISSO). The common key advantage of these two models is the fact that painstakingly handcrafted descriptors from simple material properties are not required. The main differences between the two models are (1) the use of structure information in the arbitrary size of compounds (CGCNN) and (2) limited interpretability (CGCNN) but simple and analytic relations between descriptor-property (SISSO). Using these two ML algorithms we evaluate the prediction performance on the target properties, which are band gap, formation energy, and elasticity of crystalline compounds in the database of Materials Project (MP). Moreover, to improve prediction of the properties of the materials without human bias in the selection of initial atomic features for the CGCNNs, we use Atom2Vec that provides atom representation obtained in an unsupervised manner from the materials. We also perform the predictions with the different sizes of training set to investigate the data-size dependency of the predictive models. According to the amount of dataset, the use of structural information, and the ability to identify the best descriptor with its interpretability, these algorithms showed different prediction performances. This result will enable researchers in materials discovery to gain appropriate choices and insights in various attempts to improve the prediction performance of crystalline materials’ properties.

Network-adjusted Kendall's Tau Measure for Feature Screening with Application to High-dimensional Survival Genomic Data.

In high-dimensional genetic/genomic data, the identification of genes related to clinical survival trait is a challenging and important issue. In particular, right-censored survival outcomes and contaminated biomarker data make the relevant feature screening difficult. Several independence screening methods have been developed, but they fail to account for gene-gene dependency information, and may be sensitive to outlying feature data. We improve the inverse probability-of-censoring weighted (IPCW) Kendall's tau statistic by using Google's PageRank Markov matrix to incorporate feature dependency network information. Also, to tackle outlying feature data, the nonparanormal approach transforming the feature data to multivariate normal variates are utilized in the graphical lasso procedure to estimate the network structure in feature data. Simulation studies under various scenarios show that the proposed network-adjusted weighted Kendall's tau approach leads to more accurate feature selection and survival prediction than the methods without accounting for feature dependency network information and outlying feature data. The applications on the clinical survival outcome data of diffuse large B-cell lymphoma and of The Cancer Genome Atlas lung adenocarcinoma patients demonstrate clearly the advantages of the new proposal over the alternative methods. Supplementary data are available at Bioinformatics online.

High-throughput reaction engineering to assess the oxidation stability of MAX phases

The resistance to oxidizing environments exhibited by some Mn+1AXn (MAX) phases stems from the formation of stable and protective oxide layers at high operating temperatures. The MAX phases are hexagonally arranged layered nitrides or carbides with general formula Mn+1AXn, n = 1, 2, 3, where M is early transition elements, A is A block elements, and X is C/N. Previous attempts to model and assess oxide phase stability in these systems has been limited in scope due to higher computational costs. To address the issue, we developed a machine-learning driven high-throughput framework for the fast assessment of phase stability and oxygen reactivity of 211 chemistry MAX phase M2AX. The proposed scheme combines a sure independence screening sparsifying operator-based machine-learning model in combination with grand-canonical linear programming to assess temperature-dependent Gibbs free energies, reaction products, and elemental chemical activity during the oxidation of MAX phases. The thermodynamic stability, and chemical activity of constituent elements of Ti2AlC with respect to oxygen were fully assessed to understand the high-temperature oxidation behavior. The predictions are in good agreement with oxidation experiments performed on Ti2AlC. We were also able to explain the metastability of Ti2SiC, which could not be synthesized experimentally due to higher stability of competing phases. For generality of the proposed approach, we discuss the oxidation mechanism of Cr2AlC. The insights of oxidation behavior will enable more efficient design and accelerated discovery of MAX phases with maintained performance in oxidizing environments at high temperatures.

A 10-gene-methylation-based signature for prognosis prediction of colorectal cancer

BackgroundColorectal cancer (CRC) is a common malignant tumor of digestive tract which has high incidence and mortality rates. Accurate prognosis prediction of CRC patients is pivotal to reduce the mortality and disease burden. MethodsIn this study, we comprehensively analyzed the gene expression and methylation data of CRC samples from The Cancer Genome Atlas (TCGA). Differential expression genes (DEGs) and methylation CpGs (DMCs) in tumor tissues compared with adjacent normal tissues of CRC were first identified. Functional enrichment analysis of DEGs and DMCs was performed by Database for Annotation, Visualization and Integrated Discovery (DAVID). Spearman correlation analysis was used to screen DMCs that negatively correlated with gene expressions which were subsequently applied to sure independence screening (SIS) along with stepwise regression for screening optimal CpGs for CRC prognosis prediction model construction by Cox regression analysis. ResultsWe identified a total of 1774 DEGs (663 upregulated and 1111 downregulated) and 11,975 DMCs (7385 hypermethylated and 4590 hypomethylated) in CRC tumor samples compared with adjacent normal samples. The hypermethylated loci were mainly located on CpG island, while the hypomethylated loci were mainly located on N-shore. Spearman correlation analysis screened 321 DMCs that negatively correlated with expressions of their annotated genes. Cox regression model consist of 10 CpGs was finally established which could effectively stratified CRC patients that exhibited significantly different overall survival probability independent of age, gender, and pathological staging. ConclusionWe established a prognosis prediction model based on 10 methylation sites, which could evaluate the prognosis of CRC patients.

Feature Screening for Network Autoregression Model.

Network analysis has drawn great attention in recent years. It is applied to a wide range disciplines. These include but are not limited to social science, finance and genetics. It is typical that one collects abundant covariates along the response variable in practice. Since the network structure makes the responses at different nodes no longer independent, existing screening methods may not perform well for network data. We propose a network-based sure independence screening (NW-SIS) method. This approach explicitly takes the network structure into consideration. The strong screening consistency property of the NW-SIS is rigorously established. We further investigated the estimation of the network effect and establish the -consistency of the estimator. The finite sample performance of the proposed method is assessed by simulation study and illustrated by an empirical analysis of a dataset from Chinese stock market.

A note on marginal correlation based screening

AbstractIndependence screening methods such as the two‐sample t‐test and the marginal correlation based ranking are among the most widely used techniques for variable selection in ultrahigh‐dimensional data sets. In this short note, simple examples are used to demonstrate potential problems with the independence screening methods in the presence of correlated predictors. Also, an example is considered where all important variables are independent among themselves and all but one important variables are independent with the unimportant variables. Furthermore, a real data example from a genome‐wide association study is used to illustrate inferior performance of marginal correlation screening compared to another screening method.

Feature Screening for High-Dimensional Survival Data via Censored Quantile Correlation

This paper proposes a new sure independence screening procedure for high-dimensional survival data based on censored quantile correlation (CQC). This framework has two distinctive features: 1) Via incorporating a weighting scheme, our metric is a natural extension of quantile correlation (QC), considered by Li (2015), to handle high-dimensional survival data; 2) The proposed method not only is robust against outliers, but also can discover the nonlinear relationship between independent variables and censored dependent variable. Additionally, the proposed method enjoys the sure screening property under certain technical conditions. Simulation results demonstrate that the proposed method performs competitively on survival datasets of high-dimensional predictors.

Dynamic tilted current correlation for high dimensional variable screening

Variable screening is a commonly used procedure in high dimensional data analysis to reduce dimensionality and ensure the applicability of available statistical methods. Such a procedure is complicated and computationally burdensome because spurious correlations commonly exist among predictor variables, while important predictor variables may not have large marginal correlations with the response variable. To circumvent these issues, in this paper, we develop a new screening technique, the “dynamic tilted current correlation screening” (DTCCS), for high dimensional variable screening. DTCCS is capable of selecting the most relevant predictors within a finite number of steps, and takes the popularly used sure independence screening (SIS) method and the high-dimensional ordinary least squares projection (HOLP) approach as its special cases. The DTCCS technique has sure screening and consistency properties which are justified theoretically and demonstrated numerically. A real example of gene expression data is analyzed using the proposed DTCCS procedure.

High-throughput prediction of the carrier relaxation time via data-driven descriptor

It has been demonstrated that many promising thermoelectric materials, such as tetradymite compounds are also three-dimensional topological insulators. In both cases, a fundamental question is the evaluation of carrier relaxation time, which is usually a rough task due to the complicated scattering mechanisms. Previous works using the simple deformation potential theory or considering complete electron-phonon coupling are, however, restricted to small systems. By adopting a data-driven method named SISSO (Sure Independence Screening and Sparsifying Operator) with the training data obtained via deformation potential theory, we propose an efficient and physically interpretable descriptor to evaluate the relaxation time, using tetradymites as prototypical examples. Without any input from first-principles calculations, the descriptor contains only several elemental properties of the constituent atoms, and could be utilized to quickly and reliably predict the carrier relaxation time of a substantial number of tetradymites with arbitrary stoichiometry.

A model-free conditional screening approach via sufficient dimension reduction

ABSTRACT Conditional variable screening arises when researchers have prior information regarding the importance of certain predictors. It is natural to consider feature screening methods conditioning on these known important predictors. Barut, E., Fan, J., and Verhasselt, A. [(2016), ‘Conditional Sure Independence Screening’, Journal of the American Statistical Association, 111, 1266–1277] proposed conditional sure independence screening (CSIS) to address this issue under the context of generalised linear models. While CSIS outperforms the marginal screening method when few of the factors are known to be important and/or significant correlations between some of the factors exist, unfortunately, CSIS is model based and might fail when the models are misspecified. We propose a model-free conditional screening method under the framework of sufficient dimension reduction for ultrahigh dimensional statistical problems. Numerical studies show our method easily beats CSIS for nonlinear models and performs comparable to CSIS for (generalised) linear models. Sure screening consistency property for our method is proved.

Which bridge estimator is the best for variable selection?

We study the problem of variable selection for linear models under the high-dimensional asymptotic setting, where the number of observations $n$ grows at the same rate as the number of predictors $p$. We consider two-stage variable selection techniques (TVS) in which the first stage uses bridge estimators to obtain an estimate of the regression coefficients, and the second stage simply thresholds this estimate to select the “important” predictors. The asymptotic false discovery proportion ($\operatorname{AFDP}$) and true positive proportion (ATPP) of these TVS are evaluated. We prove that for a fixed ATPP, in order to obtain a smaller $\operatorname{AFDP}$, one should pick a bridge estimator with smaller asymptotic mean square error in the first stage of TVS. Based on such principled discovery, we present a sharp comparison of different TVS, via an in-depth investigation of the estimation properties of bridge estimators. Rather than “orderwise” error bounds with loose constants, our analysis focuses on precise error characterization. Various interesting signal-to-noise ratio and sparsity settings are studied. Our results offer new and thorough insights into high-dimensional variable selection. For instance, we prove that a TVS with Ridge in its first stage outperforms TVS with other bridge estimators in large noise settings; two-stage LASSO becomes inferior when the signal is rare and weak. As a by-product, we show that two-stage methods outperform some standard variable selection techniques, such as $\operatorname{LASSO}$ and Sure Independence Screening, under certain conditions.

Asset selection based on high frequency Sharpe ratio

In portfolio choice problems, the classical Mean–Variance model in Markowitz (1952) relies heavily on the covariance structure among assets. As the number and types of assets increase rapidly, traditional methods to estimate the covariance matrix and its inverse suffer from the common issues in high or ultra-high dimensional analysis. To avoid the issue of estimating the covariance matrix with high or ultra-high dimensional data, we propose a fast procedure to reduce dimension based on a new risk/return measure constructed from intra-day high frequency data and select assets via Dependent Sure Explained Variability and Independence Screening (D-SEVIS). While most feature screening methods assume i.i.d. samples, by nature of our data, we make contribution to studying D-SEVIS for samples with serial correlation, specifically, for the stationary α-mixing processes. Under α-mixing condition, we prove that D-SEVIS satisfies sure screening property and ranking consistency property. More importantly, with the assets selected through D-SEVIS, we will build a portfolio that earns more excess return compared with several existing portfolio allocation methods. We illustrate this advantage of our asset selection method with the real data from the stock market.

Penalized linear regression with high-dimensional pairwise screening

In variable selection, most existing screening methods focus on marginal effects and ignore dependence between covariates. To improve the performance of selection, we incorporate pairwise effects in covariates for screening and penalization. We achieve this by studying the asymptotic distribution of the maximal absolute pairwise sample correlation among independent covariates. The novelty of the theory is in that the convergence is with respect to the dimensionality $p$, and is uniform with respect to the sample size $n$. Moreover, we obtain an upper bound for the maximal pairwise R squared when regressing the response onto two different covariates. Based on these extreme value results, we propose a screening procedure to detect covariates pairs that are potentially correlated and associated with the response. We further combine the pairwise screening with Sure Independence Screening and develop a new regularized variable selection procedure. Numerical studies show that our method is very competitive in terms of both prediction accuracy and variable selection accuracy.

Adaptive lasso with weights based on normalized filtering scores in molecular big data

The molecular big data are highly correlated, and numerous genes are not related. The various classification methods performance mainly rely on the selection of significant genes. Sparse regularized regression (SRR) models using the least absolute shrinkage and selection operator (lasso) and adaptive lasso (alasso) are popularly used for gene selection and classification. Nevertheless, it becomes challenging when the genes are highly correlated. Here, we propose a modified adaptive lasso with weights using the ranking-based feature selection (RFS) methods capable of dealing with the highly correlated gene expression data. Firstly, an RFS methods such as Fisher’s score (FS), Chi-square (CS), and information gain (IG) are employed to ignore the unimportant genes and the top significant genes are chosen through sure independence screening (SIS) criteria. The scores of the ranked genes are normalized and assigned as proposed weights to the alasso method to obtain the most significant genes that were proven to be biologically related to the cancer type and helped in attaining higher classification performance. With the synthetic data and real application of microarray data, we demonstrated that the proposed alasso method with RFS methods is a better approach than the other known methods such as alasso with filtering such as ridge and marginal maximum likelihood estimation (MMLE), lasso and alasso without filtering. The metrics of accuracy, area under the receiver operating characteristics curve (AUROC), and geometric mean (GM-mean) are used for evaluating the performance of the models.

PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data

Scientists have long quantified empirical observations by developing mathematical models that characterize the observations, have some measure of interpretability, and are capable of making predictions. Dynamical systems models in particular have been widely used to study, explain, and predict system behavior in a wide range of application areas, with examples ranging from Newton’s laws of classical mechanics to the Michaelis-Menten kinetics for modeling enzyme kinetics. While governing laws and equations were traditionally derived by hand, the current growth of available measurement data and resulting emphasis on data-driven modeling motivates algorithmic approaches for model discovery. A number of such approaches have been developed in recent years and have generated widespread interest, including Eureqa (Schmidt & Lipson, 2009), sure independence screening and sparsifying operator (Ouyang, Curtarolo, Ahmetcik, Scheffler, & Ghiringhelli, 2018), and the sparse identification of nonlinear dynamics (SINDy) (Brunton, Proctor, & Kutz, 2016). Maximizing the impact of these model discovery methods requires tools to make them widely accessible to scientists across domains and at various levels of mathematical expertise.

Projection correlation between scalar and vector variables and its use in feature screening with multi-response data

In this article, we introduce a new methodology to perform feature screening for ultrahigh dimensional data with multivariate responses. Several extant screening procedures are available for multivariate responses, but they may be adversely affected by heavy-tailed observations or the dimension of multivariate responses. In order to attack these challenges, we first introduce a nonparametric coefficient, called projection correlation, to measure the departure of dependence between a scalar variable X and a vector variable . It takes values between zero and one, does not require any moment conditions on X and , and is zero if and only if X and are independent. Based on its estimation that has desirable theoretical properties, such as algebraic simplicity and consistency, we present a novel sure independence screening procedure, which enjoys the desirable sure screening property. Numerical results demonstrate the effectiveness of the proposed procedure in comparison with the existing counterparts.

A Two-Stage Mutual Information Based Bayesian Lasso Algorithm for Multi-Locus Genome-Wide Association Studies.

Genome-wide association study (GWAS) has turned out to be an essential technology for exploring the genetic mechanism of complex traits. To reduce the complexity of computation, it is well accepted to remove unrelated single nucleotide polymorphisms (SNPs) before GWAS, e.g., by using iterative sure independence screening expectation-maximization Bayesian Lasso (ISIS EM-BLASSO) method. In this work, a modified version of ISIS EM-BLASSO is proposed, which reduces the number of SNPs by a screening methodology based on Pearson correlation and mutual information, then estimates the effects via EM-Bayesian Lasso (EM-BLASSO), and finally detects the true quantitative trait nucleotides (QTNs) through likelihood ratio test. We call our method a two-stage mutual information based Bayesian Lasso (MBLASSO). Under three simulation scenarios, MBLASSO improves the statistical power and retains the higher effect estimation accuracy when comparing with three other algorithms. Moreover, MBLASSO performs best on model fitting, the accuracy of detected associations is the highest, and 21 genes can only be detected by MBLASSO in Arabidopsis thaliana datasets.