Ultrahigh-dimensional Data Research Articles

Model average feature screening for ultrahigh dimensional data with responses missing at random

Model average feature screening for ultrahigh dimensional data with responses missing at random

Robust Feature Screening via Distance Correlation for Ultrahigh Dimensional Data With Responses Missing at Random

Robust Feature Screening via Distance Correlation for Ultrahigh Dimensional Data With Responses Missing at Random

Unified model-free interaction screening via CV-entropy filter

For many practical high-dimensional problems, interactions have been increasingly found to play important roles beyond main effects. A representative example is gene-gene interaction. Joint analysis, which analyzes all interactions and main effects in a single model, can be seriously challenged by high dimensionality. For high-dimensional data analysis in general, marginal screening has been established as effective for reducing computational cost, increasing stability, and improving estimation/selection performance. Most of the existing marginal screening methods are designed for the analysis of main effects only. The existing screening methods for interaction analysis are often limited by making stringent model assumptions, lacking robustness, and/or requiring predictors to be continuous (and hence lacking flexibility). A unified marginal screening approach tailored to interaction analysis is developed, which can be applied to regression, classification, and survival analysis. Predictors are allowed to be continuous and discrete. The proposed approach is built on Coefficient of Variation (CV) filters based on information entropy. Statistical properties are rigorously established. It is shown that the CV filters are almost insensitive to the distribution tails of predictors, correlation structure among predictors, and sparsity level of signals. An efficient two-stage algorithm is developed to make the proposed approach scalable to ultrahigh-dimensional data. Simulations and the analysis of TCGA LUAD data further establish the practical superiority of the proposed approach.

Model-free conditional screening for ultrahigh-dimensional survival data via conditional distance correlation.

How to select the active variables that have significant impact on the event of interest is a very important and meaningful problem in the statistical analysis of ultrahigh-dimensional data. In many applications, researchers often know that a certain set of covariates are active variables from some previous investigations and experiences. With the knowledge of the important prior knowledge of active variables, we propose a model-free conditional screening procedure for ultrahigh dimensional survival data based on conditional distance correlation. The proposed procedure can effectively detect the hidden active variables that are jointly important but are weakly correlated with the response. Moreover, it performs well when covariates are strongly correlated with each other. We establish the sure screening property and the ranking consistency of the proposed method and conduct extensive simulation studies, which suggests that the proposed procedure works well for practical situations. Then, we illustrate the new approach through a real dataset from the diffuse large-B-cell lymphomastudy S1.

A Flexibly Conditional Screening Approach via a Nonparametric Quantile Partial Correlation

Considering the influence of conditional variables is crucial to statistical modeling, ignoring this may lead to misleading results. Recently, Ma, Li and Tsai proposed the quantile partial correlation (QPC)-based screening approach that takes into account conditional variables for ultrahigh dimensional data. In this paper, we propose a nonparametric version of quantile partial correlation (NQPC), which is able to describe the influence of conditional variables on other relevant variables more flexibly and precisely. Specifically, the NQPC firstly removes the effect of conditional variables via fitting two nonparametric additive models, which differs from the conventional partial correlation that fits two parametric models, and secondly computes the QPC of the resulting residuals as NQPC. This measure is very useful in the situation where the conditional variables are highly nonlinearly correlated with both the predictors and response. Then, we employ this NQPC as the screening utility to do variable screening. A variable screening procedure based on NPQC (NQPC-SIS) is proposed. Theoretically, we prove that the NQPC-SIS enjoys the sure screening property that, with probability going to one, the selected subset can recruit all the truly important predictors under mild conditions. Finally, extensive simulations and an empirical application are carried out to demonstrate the usefulness of our proposal.

Model Selection for High Dimensional Nonparametric Additive Models via Ridge Estimation

In ultrahigh dimensional data analysis, to keep computational performance well and good statistical properties still working, nonparametric additive models face increasing challenges. To overcome them, we introduce a methodology of model selection for high dimensional nonparametric additive models. Our approach is to propose a novel group screening procedure via nonparametric smoothing ridge estimation (GRIE) to find the importance of each covariate. It is then combined with the sure screening property of GRIE and the model selection property of extended Bayesian information criteria (EBIC) to select the suitable sub-models in nonparametric additive models. Theoretically, we establish the strong consistency of model selection for the proposed method. Extensive simulations and two real datasets illustrate the outstanding performance of the GRIE-EBIC method.

Regularized Linear Programming Discriminant Rule with Folded Concave Penalty for Ultrahigh-Dimensional Data

We propose the regularized linear programming discriminant (LPD) rule with folded concave penalty in the ultrahigh-dimensional regime. We use the local linear approximation (LLA) algorithm to redirect the model with folded concave penalty to a weighted ℓ 1 model. The strong oracle property of the solution constructed by the one-step local linear approximation (LLA) algorithm is verified. In addition, we propose efficient and parallelizable algorithms based on feature space split to address the computational challenges due to ultrahigh dimensionality. The proposed feature-split algorithm is compared to existing methods by both numerical simulations and applications to real data examples. The numerical comparisons suggest that the proposed method works well for ultrahigh dimensions, while the linear programming solver and alternating direction method of multiplier (ADMM) algorithm may fail for such high dimensions. Supplementary materials for this article are available online.

A post-screening diagnostic study for ultrahigh dimensional data

We propose a consistent lack-of-fit test to assess whether replacing the original ultrahigh dimensional covariates with a given number of linear combinations results in a loss of regression information. To attenuate the spurious correlations that may inflate type-I error rates in high dimensions, we suggest to randomly split the observations into two parts. In the first part, we screen out as many irrelevant covariates as possible. This screening step helps to reduce the ultrahigh dimensionality to a moderate scale. In the second part, we perform a lack-of-fit test for conditional independence in the context of sufficient dimension reduction. In case that some important covariates are missed with a non-ignorable probability in the first screening stage, we introduce a multiple splitting procedure. We further propose a new statistic to test for conditional independence, which is shown to be n-consistent under the null and root-n-consistent under the alternative. We develop a consistent bootstrap procedure to approximate the asymptotic null distribution. The performances of our proposal are evaluated through comprehensive simulations and an empirical analysis of GDP data.

High-dimensional variable screening through kernel-based conditional mean dependence

This article proposes a model-free feature screening method for ultrahigh-dimensional data. The proposed method is built on the called kernel-based conditional mean dependence, which is defined based on the reproducing kernel embedding and the Hilbert–Schmidt norm of a tensor operator. Theoretically, both sure screening and rank consistency properties are established under weak assumptions. Through an extensive simulation study as well as a real data analysis, it is illustrated that the statistics generated by other kernels in distance kernel family are more sensitive to feature screening in ultra-high dimensions.

Block-diagonal precision matrix regularization for ultra-high dimensional data

A method that estimates the precision matrix of multiple variables in the extreme scope of “ultrahigh dimension” and “small sample-size” is proposed. Initially, a covariance column-wise screening method is provided in order to identify a small sub-group, which are significantly correlated, from thousands and even millions of variables. Then, a regularization of block-diagonal covariance structure of the thousands or millions of variables is imposed, in which only the covariances of variables in that small sub-group are retained and all others vanish. It is further proven that under some mild conditions the vital sub-group identified by the covariance column-wise screening method is consistent. A major advantage of the proposed method is its efficiency - it produces a reliable precision matrix estimator for thousands of variables within a few of seconds while the existing methods take at least several hours and even so still yield inaccurate estimators. Empirical data studies and numerical simulations show that the proposed precision matrix estimation greatly outperforms existing methods in the sense of taking much less computing time and resulting in much more accurate estimation when dealing with ultrahigh dimensional data.

A screening method for ultra-high dimensional features with overlapped partition structures.

Ultra-high dimensional data, such as gene and neuroimaging data, are becoming increasingly important in biomedical science. Identifying important biomarkers from the huge number of features can help us gain better insights into further researches. Variable screening is an efficient tool to achieve this goal under the large scale cases, which reduces the dimension of features into a moderate size by removing the major part of inactive ones. Developing novel variable screening methods for high-dimensional features with group structures is challenging, especially under the overlapped cases. For example, the huge-scaled genes usually can be partitioned into hundreds of pathways according to background knowledge. One primary characteristic for this type of data is that many genes may appear across more than one pathway, which means that different pathways are overlapped. However, existing variable screening methods only could deal with disjoint group structure cases. To fill this gap, we propose a novel variable screening method for the generalized linear model by incorporating overlapped partition structures with theoretical guarantee. Besides the sure screening property, we also test the performance of the proposed method through a series of numerical studies and apply it to statistical analysis of a breast cancer data.

Feature screening of ultrahigh dimensional longitudinal data based on the C‐statistic

AbstractThis paper considers the feature screening method for the ultrahigh dimensional semiparametric linear models with longitudinal data. The C‐statistic which measures the rank concordance between predictors and outcomes is generalized to the longitudinal data. On the basis of C‐statistic and the score equation theory, we propose a feature screening method named LCSIS. Based on the smoothed technique and the score equations, the proposed estimating screening procedure is easy to compute and satisfies the feature screening consistency. Furthermore, Monte Carlo simulation studies and a real data application are conducted to examine the finite sample performance of the proposed procedure.

Classification and prediction for multi-cancer data with ultrahigh-dimensional gene expressions.

Analysis of gene expression data is an attractive topic in the field of bioinformatics, and a typical application is to classify and predict individuals’ diseases or tumors by treating gene expression values as predictors. A primary challenge of this study comes from ultrahigh-dimensionality, which makes that (i) many predictors in the dataset might be non-informative, (ii) pairwise dependence structures possibly exist among high-dimensional predictors, yielding the network structure. While many supervised learning methods have been developed, it is expected that the prediction performance would be affected if impacts of ultrahigh-dimensionality were not carefully addressed. In this paper, we propose a new statistical learning algorithm to deal with multi-classification subject to ultrahigh-dimensional gene expressions. In the proposed algorithm, we employ the model-free feature screening method to retain informative gene expression values from ultrahigh-dimensional data, and then construct predictive models with network structures of selected gene expression accommodated. Different from existing supervised learning methods that build predictive models based on entire dataset, our approach is able to identify informative predictors and dependence structures for gene expression. Throughout analysis of a real dataset, we find that the proposed algorithm gives precise classification as well as accurate prediction, and outperforms some commonly used supervised learning methods.

Fast Lasso-type safe screening for Fine-Gray competing risks model with ultrahigh dimensional covariates.

The Fine-Gray proportional sub-distribution hazards (PSH) model is among the most popular regression model for competing risks time-to-event data. This article develops a fast safe feature elimination method, named PSH-SAFE, for fitting the penalized Fine-Gray PSH model with a Lasso (or adaptive Lasso) penalty. Our PSH-SAFE procedure is straightforward to implement, fast, and scales well to ultrahigh dimensional data. We also show that as a feature screening procedure, PSH-SAFE is safe in a sense that the eliminated features are guaranteed to be inactive features in the original Lasso (or adaptive Lasso) estimator for the penalized PSH model. We evaluate the performance of the PSH-SAFE procedure in terms of computational efficiency, screening efficiency and safety, run-time, and prediction accuracy on multiple simulated datasets and a real bladder cancer data. Our empirical results show that the PSH-SAFE procedure possesses desirable screening efficiency and safety properties and can offer substantially improved computational efficiency as well as similar or better prediction performance in comparison to their baseline competitors.

High-dimensional robust approximated [formula omitted]-estimators for mean regression with asymmetric data

Asymmetry along with heteroscedasticity or contamination often occurs with the growth of data dimensionality. In ultra-high dimensional data analysis, such irregular settings are usually overlooked for both theoretical and computational convenience. In this paper, we establish a framework for estimation in high-dimensional regression models using Penalized Robust Approximated quadratic M-estimators (PRAM). This framework allows general settings such as random errors lack symmetry and homogeneity, or covariates are not sub-Gaussian. To reduce the possible bias caused by data’s irregularity in mean regression, PRAM adopts a loss function with an adaptive robustification parameter. Theoretically, we first show that, in the ultra-high dimension setting, PRAM estimators have local estimation consistency at the minimax rate enjoyed by the LS-Lasso. Then we show that PRAM with an appropriate non-convex penalty in fact agrees with the local oracle solution, and thus obtain its oracle property. Computationally, we compare the performances of six PRAM estimators (Huber, Tukey’s biweight or Cauchy loss function combined with Lasso or MCP penalty function). Our simulation studies and real data analysis demonstrate satisfactory finite sample performances of the PRAM estimator under different irregular settings.

Model-Free Conditional Feature Screening with FDR Control

In this article, we propose a model-free conditional feature screening method with false discovery rate (FDR) control for ultra-high dimensional data. The proposed method is built upon a new measure of conditional independence. Thus, the new method does not require a specific functional form of the regression function and is robust to heavy-tailed responses and predictors. The variables to be conditional on are allowed to be multivariate. The proposed method enjoys sure screening and ranking consistency properties under mild regularity conditions. To control the FDR, we apply the Reflection via Data Splitting method and prove its theoretical guarantee using martingale theory and empirical process techniques. Simulated examples and real data analysis show that the proposed method performs very well compared with existing works. Supplementary materials for this article are available online.

A Combined Feature Screening Approach of Random Forest and Filterbased Methods for Ultra-high Dimensional Data

Background: Various feature (variable) screening approaches have been proposed in the past decade to mitigate the impact of ultra-high dimensionality in classification and regression problems, including filter based methods such as sure independence screening, and wrapper based methods such as random forest. However, the former type of methods rely heavily on strong modelling assumptions while the latter ones requires an adequate sample size to make the data speak for themselves. These requirements can seldom be met in biochemical studies in cases where we have only access to ultra-high dimensional data with a complex structure and a small number of observations. Objective: In this research, we want to investigate the possibility of combining both filter based screening methods and random forest based screening methods in the regression context. Method: We have combined four state-of-art filter approaches, namely, sure independence screening (SIS), robust rank correlation based screening (RRCS), high dimensional ordinary least squares projection (HOLP) and a model free sure independence screening procedure based on the distance correlation (DCSIS) from the statistical community with a random forest based Boruta screening method from the machine learning community for regression problems. Result: Among all the combined methods, RF-DCSIS performs better than the other methods in terms of screening accuracy and prediction capability on the simulated scenarios and real benchmark datasets. Conclusion: By empirical study from both extensive simulation and real data, we have shown that both filter based screening and random forest based screening have their pros and cons, while a combination of both may lead to a better feature screening result and prediction capability.

Model-free feature screening for ultrahigh dimensional data via a Pearson chi-square based index

For ultrahigh dimensional data, we propose a model-free marginal feature screening procedure, which can handle continuous, categorical and discrete response variables, based on the integral Pearson chi-square (IPC) index. The IPC index can be regarded as an extension of the AD index studied by He et al. [A modified mean-variance feature-screening procedure for ultrahigh-dimensional discriminant analysis. Comput Stat Data Anal. 2019;137:155–169]. When the response variable is categorical, we extend He et al.'s work to the case of allowing a diverging number of response categories. However, the IPC index is difficult to estimate when the response is continuous. Thus we modify it and define the fused IPC index using the slice-and-fuse technique. Our feature screening procedure ranking the IPC or fused IPC index is robust to heavy-tailed features and outliers. The sure screening properties and the ranking consistency properties are established for both categorical and continuous responses under mild conditions. The finite sample performance of the proposed procedure is demonstrated through various numerical simulations and two real data applications.

Feature screening with latent responses.

A novel feature screening method is proposed to examine the correlation between latent responses and potential predictors in ultrahigh-dimensional data analysis. First, a confirmatory factor analysis (CFA) model is used to characterize latent responses through multiple observed variables. The expectation-maximization algorithm is employed to estimate the parameters in the CFA model. Second, R-Vector (RV) correlation is used to measure the dependence between the multivariate latent responses and covariates of interest. Third, a feature screening procedure is proposed on the basis of an unbiased estimator of the RV coefficient. The sure screening property of the proposed screening procedure is established under certain mild conditions. Monte Carlo simulations are conducted to assess the finite-sample performance of the feature screening procedure. The proposed method is applied to an investigation of the relationship between psychological well-being and the humangenome.

Epistasis Detection via the Joint Cumulant

Selecting influential non-linear interactive features from ultrahigh-dimensional data has been an important task in various fields. However, statistical accuracy and computational feasibility are the two biggest concerns when billions of interactive pairs are screened in practice. Many extant feature screening approaches are either focused on only main effects or heavily rely on heredity structure, hence rendering them ineffective in a scenario presenting strong interactive but weak main effects. In this article, we propose a new interaction screening procedure based on joint cumulant (named JCI-SIS). We show that the proposed procedure has strong sure screening consistency and is theoretically sound to support its performance. Finite sample simulation studies designed for both continuous and categorical predictors are performed to demonstrate the versatility and practicability of the JCI-SIS method. We further illustrate the power of JCI-SIS by applying it to screen 27,554,602,881 interaction pairs involving 234,754 single-nucleotide polymorphisms for 4098 subjects collected from polycystic ovary syndrome patients and healthy controls.