High-dimensional integrative analysis with homogeneity and sparsity recovery

Abstract

This paper studies integrative analysis of multiple units in the context of high-dimensional linear regression. We consider the case where a fraction of the covariates have different effects on the responses across various units, e.g., some coefficients are the same for all the units, while others have grouping structures. We propose a least squares approach, combined with a difference penalty term to penalize the difference between any two units’ coefficients of the same covariate for identifying latent grouping structure, as well as a common sparsity penalty to detect important covariates. Without the need to know the grouping structure of every variable across the data units and the sparsity construction within the variables, the proposed double penalized procedure can automatically identify the covariates with heterogeneous effects, covariates with homogeneous effects, and recover the sparsity, the grouping structures of the heterogeneous covariates, and provide estimates of all regression coefficients simultaneously. We proceed the alternating direction method of multipliers algorithm (ADMM) through effectively utilizing the storage and reading of the datasets, and demonstrate the convergence of the proposed procedure. We show that the proposed estimator enjoys the oracle property. Simulation studies demonstrate the good performance of the new method with finite samples, and a real data example is provided for illustration.