Ultrahigh Dimensional Covariates Research Articles

Robust model averaging prediction of longitudinal response with ultrahigh-dimensional covariates

Robust model averaging prediction of longitudinal response with ultrahigh-dimensional covariates

Decorrelated empirical likelihood for generalized linear models with high-dimensional longitudinal data

This paper focuses on decorrelated empirical likelihood-based inference for longitudinal data with ultrahigh-dimensional covariates. The primary issues we aim to address involve parameter estimation and hypothesis testing for a low-dimensional parameter of interest. Under the framework of the generalized linear model, we initially consider the within-subject correlation by linearizing the precision matrix with certain known matrices, which retains optimality even if the working correlated structure is misspecified. Coupled with the decorrelated matrix, we then eliminate the influence of nuisance parameters on the estimation procedure. The proposed approach not only yields more efficient estimators compared to generalized decorrelated estimating equations but also shares the same asymptotic variance as quadratic decorrelated inference function based methods. Furthermore, we define the decorrelated empirical log-likelihood ratio test statistic to assess the significance of regression coefficients. Finally, to evaluate the performance of the proposed procedure, we conduct simulation studies and apply it to a real data example.

A new covariate selection strategy for high dimensional data in causal effect estimation with multivariate treatments

Selection of covariates is crucial in the estimation of average treatment effects given observational data with high or even ultra-high dimensional pretreatment variables. Existing methods for this problem typically assume sparse linear models for both outcome and univariate treatment, and cannot handle situations with ultra-high dimensional covariates. In this paper, we propose a new covariate selection strategy called double screening prior adaptive lasso (DSPAL) to select confounders and predictors of the outcome for multivariate treatments, which combines the adaptive lasso method with the marginal conditional (in)dependence prior information to select target covariates, in order to eliminate confounding bias and improve statistical efficiency. The distinctive features of our proposal are that it can be applied to high-dimensional or even ultra-high dimensional covariates for multivariate treatments, and can deal with the cases of both parametric and nonparametric outcome models, which makes it more robust compared to other methods. Our theoretical analyses show that the proposed procedure enjoys the sure screening property, the ranking consistency property and the variable selection consistency. Through a simulation study, we demonstrate that the proposed approach selects all confounders and predictors consistently and estimates the multivariate treatment effects with smaller bias and mean squared error compared to several alternatives under various scenarios. In real data analysis, the method is applied to estimate the causal effect of a three-dimensional continuous environmental treatment on cholesterol level and enlightening results are obtained.

Sure Joint Screening for High Dimensional Cox's Proportional Hazards Model Under the Case-Cohort Design.

This study develops a sure joint feature screening method for the case-cohort design with ultrahigh-dimensional covariates. Our method is based on a sparsity-restricted Cox proportional hazards model. An iterative reweighted hard thresholding algorithm is proposed to approximate the sparsity-restricted, pseudo-partial likelihood estimator for joint screening. We rigorously show that our method possesses the sure screening property, with the probability of retaining all relevant covariates tending to 1 as the sample size goes to infinity. Our simulation results demonstrate that the proposed procedure has substantially improved screening performance over some existing feature screening methods for the case-cohort design, especially when some covariates are jointly correlated, but marginally uncorrelated, with the event time outcome. A real data illustration is provided using breast cancer data with high-dimensional genomic covariates. We have implemented the proposed method using MATLAB and made it available to readers through GitHub.

A post-screening diagnostic study for ultrahigh dimensional data

We propose a consistent lack-of-fit test to assess whether replacing the original ultrahigh dimensional covariates with a given number of linear combinations results in a loss of regression information. To attenuate the spurious correlations that may inflate type-I error rates in high dimensions, we suggest to randomly split the observations into two parts. In the first part, we screen out as many irrelevant covariates as possible. This screening step helps to reduce the ultrahigh dimensionality to a moderate scale. In the second part, we perform a lack-of-fit test for conditional independence in the context of sufficient dimension reduction. In case that some important covariates are missed with a non-ignorable probability in the first screening stage, we introduce a multiple splitting procedure. We further propose a new statistic to test for conditional independence, which is shown to be n-consistent under the null and root-n-consistent under the alternative. We develop a consistent bootstrap procedure to approximate the asymptotic null distribution. The performances of our proposal are evaluated through comprehensive simulations and an empirical analysis of GDP data.

Deep survival forests with feature screening

Deep forest has recently received much attention, mainly from the machine learning community, because it has fewer hyper-parameters, better performance, and is easier to construct than deep neural networks. This approach, as well as its variations, are commonly used for image classification, face recognition, music classification, and other related tasks. Its potential in survival analysis, on the other hand, has yet to be determined. In this paper, we propose a deep survival forests framework for modeling right-censored data with high and ultra-high dimensional covariates. We construct the proposed framework by merging the cascade survival forest structure with a feature screening mechanism. Experimental and statistical analysis results on well-known ultra-high and high-dimensional datasets show that the proposed methodology outperforms popular machine learning-based methods such as deep neural survival networks, random survival forests and their several variants in terms of prediction accuracy.

Fast Lasso-type safe screening for Fine-Gray competing risks model with ultrahigh dimensional covariates.

The Fine-Gray proportional sub-distribution hazards (PSH) model is among the most popular regression model for competing risks time-to-event data. This article develops a fast safe feature elimination method, named PSH-SAFE, for fitting the penalized Fine-Gray PSH model with a Lasso (or adaptive Lasso) penalty. Our PSH-SAFE procedure is straightforward to implement, fast, and scales well to ultrahigh dimensional data. We also show that as a feature screening procedure, PSH-SAFE is safe in a sense that the eliminated features are guaranteed to be inactive features in the original Lasso (or adaptive Lasso) estimator for the penalized PSH model. We evaluate the performance of the PSH-SAFE procedure in terms of computational efficiency, screening efficiency and safety, run-time, and prediction accuracy on multiple simulated datasets and a real bladder cancer data. Our empirical results show that the PSH-SAFE procedure possesses desirable screening efficiency and safety properties and can offer substantially improved computational efficiency as well as similar or better prediction performance in comparison to their baseline competitors.

Non-marginal feature screening for varying coefficient competing risks model

This article is concerned with a non-marginal feature screening procedure for varying coefficient competing risks models with ultra-high dimensional covariates. The proposed method enjoys ascent property and sure screening property. Its finite-sample performances are illustrated by numerical studies.

Variable selection for partially linear models via Bayesian subset modeling with diffusing prior

Most existing methods of variable selection in partially linear models (PLM) with ultrahigh dimensional covariates are based on partial residuals, which involve a two-step estimation procedure. While the estimation error produced in the first step may have an impact on the second step, multicollinearity among predictors adds additional challenges in the model selection procedure. In this paper, we propose a new Bayesian variable selection approach for PLM. This new proposal addresses those two issues simultaneously as (1) it is a one-step method which selects variables in PLM, even when the dimension of covariates increases at an exponential rate with the sample size, and (2) the method retains model selection consistency, and outperforms existing ones in the setting of highly correlated predictors. Distinguished from existing ones, our proposed procedure employs the difference-based method to reduce the impact from the estimation of the nonparametric component, and incorporates Bayesian subset modeling with diffusing prior (BSM-DP) to shrink the corresponding estimator in the linear component. The estimation is implemented by Gibbs sampling, and we prove that the posterior probability of the true model being selected converges to one asymptotically. Simulation studies support the theory and the efficiency of our methods as compared to other existing ones, followed by an application in a study of supermarket data.

TEST OF SIGNIFICANCE FOR HIGH-DIMENSIONAL LONGITUDINAL DATA.

This paper concerns statistical inference for longitudinal data with ultrahigh dimensional covariates. We first study the problem of constructing confidence intervals and hypothesis tests for a low dimensional parameter of interest. The major challenge is how to construct a powerful test statistic in the presence of high-dimensional nuisance parameters and sophisticated within-subject correlation of longitudinal data. To deal with the challenge, we propose a new quadratic decorrelated inference function approach, which simultaneously removes the impact of nuisance parameters and incorporates the correlation to enhance the efficiency of the estimation procedure. When the parameter of interest is of fixed dimension, we prove that the proposed estimator is asymptotically normal and attains the semiparametric information bound, based on which we can construct an optimal Wald test statistic. We further extend this result and establish the limiting distribution of the estimator under the setting with the dimension of the parameter of interest growing with the sample size at a polynomial rate. Finally, we study how to control the false discovery rate (FDR) when a vector of high-dimensional regression parameters is of interest. We prove that applying the Storey (2002)'s procedure to the proposed test statistics for each regression parameter controls FDR asymptotically in longitudinal data. We conduct simulation studies to assess the finite sample performance of the proposed procedures. Our simulation results imply that the newly proposed procedure can control both Type I error for testing a low dimensional parameter of interest and the FDR in the multiple testing problem. We also apply the proposed procedure to a real data example.

Model free feature screening for ultrahigh dimensional covariates with right censored outcomes

This paper is concerned with feature screening for the ultrahigh dimensional survival data. We propose a new feature screening procedure by extending the method of Zhu et al. via inverse probability censoring weighting. The proposed procedure enjoys two appealing merits. First, it does not need to specify any model assumption between the response and the covariates. Thus, it is robust to the model mis-specification. Second, our procedure is robust in the presence of outliers or extreme values since it only uses the rank of censored outcomes. We establish the sure screening property under some regular conditions. The simulations and analysis of the real data demonstrate that our procedure exhibits favorably in comparison with the existing competitors.

Non-marginal feature screening for additive hazard model with ultrahigh-dimensional covariates

Survival data with ultrahigh-dimensional covariates have been frequently encountered in medical studies and other fields. In this article, we propose a non-marginal feature screening procedure for the additive hazard model with ultrahigh-dimensional covariates. The proposed method utilizes the joint effects between covariates, which can effectively identify active covariates that are jointly dependent but marginally independent of the response. We develop an iterative hard-thresholding (IHT) algorithm to effectively implement the proposed procedure. Its convergence properties were investigated rigorously and further proved that the proposed procedure possesses the sure screening property. We conduct Monte Carlo simulation to evaluate the finite sample performance of the proposed procedure, and demonstrate the proposed procedure through an empirical analysis of a real data example.

Sparse Learning and Structure Identification for Ultrahigh-Dimensional Image-on-Scalar Regression

This article considers high-dimensional image-on-scalar regression, where the spatial heterogeneity of covariate effects on imaging responses is investigated via a flexible partially linear spatially varying coefficient model. To tackle the challenges of spatial smoothing over the imaging response’s complex domain consisting of regions of interest, we approximate the spatially varying coefficient functions via bivariate spline functions over triangulation. We first study estimation when the active constant coefficients and varying coefficient functions are known in advance. We then further develop a unified approach for simultaneous sparse learning and model structure identification in the presence of ultrahigh-dimensional covariates. Our method can identify zero, nonzero constant, and spatially varying components correctly and efficiently. The estimators of constant coefficients and varying coefficient functions are consistent and asymptotically normal for constant coefficient estimators. The method is evaluated by Monte Carlo simulation studies and applied to a dataset provided by the Alzheimer’s Disease Neuroimaging Initiative. Supplementary materials for this article are available online.

Feature screening under missing indicator imputation with non-ignorable missing response

This article develops a model-free variable screening technique with the non-ignorable missing response in ultrahigh-dimensional data analysis. Based on the common logistic model assumption of the propensity function, a novel screening procedure is proposed by borrowing hidden information of missingness indicator such that any variable screening method for ultrahigh-dimensional covariates with full data can be applied to the non-ignorable missing response case. And it is shown that the sure screening property can be kept as long as the corresponding screening method for full data is of sure screening property. The finite sample performances of the proposed method are demonstrated via some simulations and analysis of functional neuroimaging data.

FEATURE SELECTION FOR GENERALIZED VARYING COEFFICIENT MIXED-EFFECT MODELS WITH APPLICATION TO OBESITY GWAS.

Motivated by an empirical analysis of data from a genome-wide association study on obesity, measured by the body mass index (BMI), we propose a two-step gene-detection procedure for generalized varying coefficient mixed-effects models with ultrahigh dimensional covariates. The proposed procedure selects significant single nucleotide polymorphisms (SNPs) impacting the mean BMI trend, some of which have already been biologically proven to be "fat genes." The method also discovers SNPs that significantly influence the age-dependent variability of BMI. The proposed procedure takes into account individual variations of genetic effects and can also be directly applied to longitudinal data with continuous, binary or count responses. We employ Monte Carlo simulation studies to assess the performance of the proposed method and further carry out causal inference for the selected SNPs.

Feature Screening in Ultrahigh Dimensional Generalized Varying-coefficient Models.

Generalized varying coefficient models are particularly useful for examining dynamic effects of covariates on a continuous, binary or count response. This paper is concerned with feature screening for generalized varying coefficient models with ultrahigh dimensional covariates. The proposed screening procedure is based on joint quasi-likelihood of all predictors, and therefore is distinguished from marginal screening procedures proposed in the literature. In particular, the proposed procedure can effectively identify active predictors that are jointly dependent but marginally independent of the response. In order to carry out the proposed procedure, we propose an effective algorithm and establish the ascent property of the proposed algorithm. We further prove that the proposed procedure possesses the sure screening property. That is, with probability tending to one, the selected variable set includes the actual active predictors. We examine the finite sample performance of the proposed procedure and compare it with existing ones via Monte Carlo simulations, and illustrate the proposed procedure by a real data example.

Conditional distance correlation screening for sparse ultrahigh-dimensional models

This paper is concerned with feature screening for ultrahigh-dimensional covariates under general varying-coefficient models. With the sparsity principle and based on the conditional distance correlation, we develop a new marginal feature screening procedure called CDC-SIS to select significant covariates and show that it possesses the sure screening property and ranking consistency property under some regularity conditions. The proposed procedure enjoys two appealing merits. First, the model we considered is more flexible than traditional varying-coefficients regression models, so the method can be used in a wider range of applications. Second, CDC-SIS can be used directly to deal with grouped predictor variables and multivariate responses. We assess the finite sample properties of the proposed procedure by Monte Carlo studies, and illustrate our method by an empirical analysis of a real data set. Compared with other similar works, our procedure yields better performance.

Robust feature screening for multi-response trans-elliptical regression model with ultrahigh-dimensional covariates

In this paper, we innovatively propose an extremely flexible semi-parametric regression model called Multi-response Trans-Elliptical Regression (MTER) Model, which can capture the heavy-tail characteristic and tail dependence of both responses and covariates. We investigate the feature screening procedure for the MTER model, in which Kendall’ tau-based canonical correlation estimators are proposed to characterize the correlation between each transformed predictor and the multivariate transformed responses. The main idea is to substitute the classical canonical correlation ranking index in [X. B. Kong, Z. Liu, Y. Yao and W. Zhou, Sure screening by ranking the canonical correlations, TEST 26 (2017) 1–25] by a carefully constructed non-parametric version. The sure screening property and ranking consistency property are established for the proposed procedure. Simulation results show that the proposed method is much more powerful to distinguish the informative features from the unimportant ones than some state-of-the-art competitors, especially for heavy-tailed distributions and high-dimensional response. At last, a real data example is given to illustrate the effectiveness of the proposed procedure.

A new joint screening method for right-censored time-to-event data with ultra-high dimensional covariates.

In an ultra-high dimensional setting with a huge number of covariates, variable screening is useful for dimension reduction before applying a more refined method for model selection and statistical analysis. This paper proposes a new sure joint screening procedure for right-censored time-to-event data based on a sparsity-restricted semiparametric accelerated failure time model. Our method, referred to as Buckley-James assisted sure screening (BJASS), consists of an initial screening step using a sparsity-restricted least-squares estimate based on a synthetic time variable and a refinement screening step using a sparsity-restricted least-squares estimate with the Buckley-James imputed event times. The refinement step may be repeated several times to obtain more stable results. We show that with any fixed number of refinement steps, the BJASS procedure retains all important variables with probability tending to 1. Simulation results are presented to illustrate its performance in comparison with some marginal screening methods. Real data examples are provided using a diffuse large-B-cell lymphoma (DLBCL) data and a breast cancer data. We have implemented the BJASS method using Matlab and made it available to readers through Github https://github.com/yiucla/BJASS .

Feature screening in ultrahigh-dimensional varying-coefficient Cox model

The varying-coefficient Cox model is flexible and useful for modeling the dynamic changes of regression coefficients in survival analysis. In this paper, we study feature screening for varying-coefficient Cox models in ultrahigh-dimensional covariates. The proposed screening procedure is based on the joint partial likelihood of all predictors, thus different from marginal screening procedures available in the literature. In order to carry out the new procedure, we propose an effective algorithm and establish its ascent property. We further prove that the proposed procedure possesses the sure screening property. That is, with probability tending to 1, the selected variable set includes the actual active predictors. We conducted simulations to evaluate the finite-sample performance of the proposed procedure and compared it with marginal screening procedures. A genomic data set is used for illustration purposes.