Dimensional Covariance Research Articles

Penalized Sparse Covariance Regression with High Dimensional Covariates

Covariance regression offers an effective way to model the large covariance matrix with the auxiliary similarity matrices. In this work, we propose a sparse covariance regression (SCR) approach to handle the potentially high-dimensional predictors (i.e., similarity matrices). Specifically, we use the penalization method to identify the informative predictors and estimate their associated coefficients simultaneously. We first investigate the Lasso estimator and subsequently consider the folded concave penalized estimation methods (e.g., SCAD and MCP). However, the theoretical analysis of the existing penalization methods is primarily based on i.i.d. data, which is not directly applicable to our scenario. To address this difficulty, we establish the non-asymptotic error bounds by exploiting the spectral properties of the covariance matrix and similarity matrices. Then, we derive the estimation error bound for the Lasso estimator and establish the desirable oracle property of the folded concave penalized estimator. Extensive simulation studies are conducted to corroborate our theoretical results. We also illustrate the usefulness of the proposed method by applying it to a Chinese stock market dataset.

Semi-Supervised Triply Robust Inductive Transfer Learning

In this work, we propose a Semi-supervised Triply Robust Inductive transFer LEarning (STRIFLE) approach, which integrates heterogeneous data from a label-rich source population and a label-scarce target population and uses a large amount of unlabeled data simultaneously to improve the learning accuracy in the target population. Specifically, we consider a high dimensional covariate shift setting and employ two nuisance models, a density ratio model and an imputation model, to combine transfer learning and surrogate-assisted semi-supervised learning strategies effectively and achieve triple robustness. While the STRIFLE approach assumes the target and source populations to share the same conditional distribution of outcome Y given both the surrogate features S and predictors X , it allows the true underlying model of Y⏧ X to differ between the two populations due to the potential covariate shift in S and X . Different from double robustness, even if both nuisance models are misspecified or the distribution of Y⏧ S , X is not the same between the two populations when the shifted source population and the target population share enough similarities, the triply robust STRIFLE estimator can still partially use the source population when the shifted source population and the target population share enough similarities. Moreover, it is guaranteed to be no worse than the target-only surrogate-assisted semi-supervised estimator with an additional error term from transferability detection. These desirable properties of our estimator are established theoretically and verified in finite samples via extensive simulation studies. We use the STRIFLE estimator to train a Type II diabetes polygenic risk prediction model for the African American target population by transferring knowledge from electronic health records linked genomic data observed in a larger European source population. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

On the estimation of quantile treatment effects using a semiparametric propensity score

. This article considers the estimation of quantile treatment effects under the assumption of unconfoundedness given quasi-experimental data. We propose a semiparametric single-index method to estimate the propensity score. Our approach overcomes the curse of dimensionality issue of a nonparametric propensity score and can handle a moderately large dimension of covariates. It is more flexible than the parametric propensity score and thereby alleviates the possible model misspecification problem. We derive the asymptotic distribution of the quantile treatment effect estimator that is based on the semiparametric propensity score. We also propose a consistent variance estimator and construct the confidence intervals for the QTE estimator. Monte Carlo simulation results show that the proposed estimator performs well in finite samples and the confidence intervals have adequate coverage rates. We demonstrate the usefulness of our method by applying it to a study of the quantile treatment effects of college education on income.

Robust multitask learning in high dimensions under memory constraint

AbstractWe investigate multitask learning in the context of multivariate linear regression with high dimensional covariates and heavy‐tailed noise, while under the constraint of limited memory. To tackle the computational complexity arising from the non‐smoothness of the quantile loss, we reformulate it as an equivalent least squares loss, which yields robust solutions even in the presence of heavy‐tailed noise. We incorporate a group lasso penalty into the quantile loss to produce sparse solutions, and an accelerated proximal sub‐gradient descent algorithm to speed up the computation while ensuring explicit forms for penalized solutions at each iteration. The proposed algorithm is general and can be applied to similar optimization problems. Moreover, we introduce a communication‐efficient distributed algorithm that guarantees optimal convergence rates after finite communication rounds in cases where computing resources such as memory are insufficient. We also study the theoretical properties of the resultant estimate and relax the widely used model selection consistency assumption on the initial estimate. We demonstrate the effectiveness of our proposal through extensive numerical studies.

An encoding generative modeling approach to dimension reduction and covariate adjustment in causal inference with observational studies

In this article, we develop CausalEGM, a deep learning framework for nonlinear dimension reduction and generative modeling of the dependency among covariate features affecting treatment and response. CausalEGM can be used for estimating causal effects in both binary and continuous treatment settings. By learning a bidirectional transformation between the high-dimensional covariate space and a low-dimensional latent space and then modeling the dependencies of different subsets of the latent variables on the treatment and response, CausalEGM can extract the latent covariate features that affect both treatment and response. By conditioning on these features, one can mitigate the confounding effect of the high dimensional covariate on the estimation of the causal relation between treatment and response. In a series of experiments, the proposed method is shown to achieve superior performance over existing methods in both binary and continuous treatment settings. The improvement is substantial when the sample size is large and the covariate is of high dimension. Finally, we established excess risk bounds and consistency results for our method, and discuss how our approach is related to and improves upon other dimension reduction approaches in causal inference.

Estimation and convergence rates in the distributional single index model

The distributional single index model is a semiparametric regression model in which the conditional distribution functions of a real‐valued outcome variable depend on ‐dimensional covariates through a univariate, parametric index function , and increase stochastically as increases. We propose least squares approaches for the joint estimation of and in the important case where and obtain convergence rates of , thereby improving an existing result that gives a rate of . A simulation study indicates that the convergence rate for the estimation of might be faster. Furthermore, we illustrate our methods in an application on house price data that demonstrates the advantages of shape restrictions in single index models.

Penalized weighted smoothed quantile regression for high-dimensional longitudinal data.

Quantile regression, known as a robust alternative to linear regression, has been widely used in statistical modeling and inference. In this paper, we propose a penalized weighted convolution-type smoothed method for variable selection and robust parameter estimation of the quantile regression with high dimensional longitudinal data. The proposed method utilizes a twice-differentiable and smoothed loss function instead of the check function in quantile regression without penalty, and can select the important covariates consistently using the efficient gradient-based iterative algorithms when the dimension of covariates is larger than the sample size. Moreover, the proposed method can circumvent the influence of outliers in the response variable and/or the covariates. To incorporate the correlation within each subject and enhance the accuracy of the parameter estimation, a two-step weighted estimation method is also established. Furthermore, we prove the oracle properties of the proposed method under some regularity conditions. Finally, the performance of the proposed method is demonstrated by simulation studies and two real examples.

Building a multistate model from electronic health records data for modeling long-term diabetes complications

Abstract Objective: The progression of long-term diabetes complications has led to a decreased quality of life. Our objective was to evaluate the adverse outcomes associated with diabetes based on a patient’s clinical profile by utilizing a multistate modeling approach. Methods: This was a retrospective study of diabetes patients seen in primary care practices from 2013 to 2017. We implemented a five-state model to examine the progression of patients transitioning from one complication to having multiple complications. Our model incorporated high dimensional covariates from multisource data to investigate the possible effects of different types of factors that are associated with the progression of diabetes. Results: The cohort consisted of 10,596 patients diagnosed with diabetes and no previous complications associated with the disease. Most of the patients in our study were female, White, and had type 2 diabetes. During our study period, 5928 did not develop complications, 3323 developed microvascular complications, 1313 developed macrovascular complications, and 1129 developed both micro- and macrovascular complications. From our model, we determined that patients had a 0.1334 [0.1284, .1386] rate of developing a microvascular complication compared to 0.0508 [0.0479, .0540] rate of developing a macrovascular complication. The area deprivation index score we incorporated as a proxy for socioeconomic information indicated that patients who reside in more disadvantaged areas have a higher rate of developing a complication compared to those who reside in least disadvantaged areas. Conclusions: Our work demonstrates how a multistate modeling framework is a comprehensive approach to analyzing the progression of long-term complications associated with diabetes.

Use of random integration to test equality of high dimensional covariance matrices.

Testing the equality of two covariance matrices is a fundamental problem in statistics, and especially challenging when the data are high-dimensional. Through a novel use of random integration, we can test the equality of high-dimensional covariance matrices without assuming parametric distributions for the two underlying populations, even if the dimension is much larger than the sample size. The asymptotic properties of our test for arbitrary number of covariates and sample size are studied in depth under a general multivariate model. The finite-sample performance of our test is evaluated through numerical studies. The empirical results demonstrate that our test is highly competitive with existing tests in a wide range of settings. In particular, our proposed test is distinctly powerful under different settings when there exist a few large or many small diagonal disturbances between the two covariance matrices.

Adaptive Tests for Bandedness of High-dimensional Covariance Matrices

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain bandwidth under the high-dimensional framework. Though several testing methods have been proposed in the literature, the existing tests are only powerful for some alternatives with certain sparsity levels, whereas they may not be powerful for alternatives with other sparsity structures. The goal of this paper is to propose a new test for the bandedness of high-dimensional covariance matrix, which is powerful for alternatives with various sparsity levels. The proposed new test also be used for testing the banded structure of covariance matrices of error vectors in high-dimensional factor models. Based on these statistics, a consistent bandwidth estimator is also introduced for a banded high dimensional covariance matrix. Extensive simulation studies and an application to a prostate cancer dataset from protein mass spectroscopy are conducted for evaluating the effectiveness of the proposed adaptive tests blue and bandwidth estimator for the banded covariance matrix.

Model averaging for generalized linear models in diverging model spaces with effective model size

. While plenty of frequentist model averaging methods have been proposed, existing weight selection criteria for generalized linear models (GLM) are usually based on a model size penalized Kullback-Leibler (KL) loss or simply cross-validation. In this article, when the data is generated from an exponential distribution, we propose a novel model averaging approach for GLM motivated by an asymptotically unbiased estimator of the KL loss penalized by an “effective model size” that incorporates the model misspecification. When all the candidate models are misspecified, the proposed method achieves asymptotic optimality while allowing both the number of candidate models and the dimension of covariates to diverging. Furthermore, when correct models are included in the candidate model set, we prove that the weight of wrong candidate models converges to zero, and hence the weighted regression coefficient estimator is consistent. Simulation studies and two real-data examples demonstrate the advantage of our new method over the existing frequentist model averaging methods.

Robust estimators of functional single index models for longitudinal data

Functional single index model is a powerful model for longitudinal data, which effectively reduces the dimension of covariates. Inspired by robust calibration of computer models, we propose a robust estimator based on Huber loss for functional single index model with fixed dimension parameter. Furthermore, we also consider the case of high dimension parameter, and proposed a robust L 1 estimator of the unknown parameter based on adaptive lasso penalty. Theoretical properties including the asymptotic and non asymptotic results have also been investigated. Numerical studies including simulated experiments and an application to AIDS data verify the validity of the proposed estimators.

A new approach for ultrahigh-dimensional covariance matrix estimation

We propose a method for estimating ultrahigh dimensional covariance matrix without the Gaussian distribution and the order of variables assumptions. More specifically, by combining the modified Cholesky decomposition (MCD) and refitted cross validation (RCV), a permutation invariant method called CovPRCV is developed to estimate covariance matrix under the “Permutation-Average” framework. The employment of RCV procedure attenuates significantly spurious correlation in the ultrahigh dimensional data. We derive the consistency of proposed estimator under the Frobenius norm without requiring banded structure and normal distribution. The finite-sample performance is assessed via simulation studies, which indicate that the proposed method is promising compared with its competitors in many interesting scenarios. We also apply the proposed method to analyze a prostate dataset.

Analysis of length-biased and partly interval-censored survival data with mismeasured covariates.

In this paper, we analyze the length-biased and partly interval-censored data, whose challenges primarily come from biased sampling and interfere induced by interval censoring. Unlike existing methods that focus on low-dimensional data and assume the covariates to be precisely measured, sometimes researchers may encounter high-dimensional data subject to measurement error, which are ubiquitous in applications and make estimation unreliable. To address those challenges, we explore a valid inference method for handling high-dimensional length-biased and interval-censored survival data with measurement error in covariates under the accelerated failure time model. We primarily employ the SIMEX method to correct for measurement error effects and propose the boosting procedure to do variable selection and estimation. The proposed method is able to handle the case that the dimension of covariates is larger than the sample size and enjoys appealing features that the distributions of the covariates are leftunspecified.

A new covariate selection strategy for high dimensional data in causal effect estimation with multivariate treatments

Selection of covariates is crucial in the estimation of average treatment effects given observational data with high or even ultra-high dimensional pretreatment variables. Existing methods for this problem typically assume sparse linear models for both outcome and univariate treatment, and cannot handle situations with ultra-high dimensional covariates. In this paper, we propose a new covariate selection strategy called double screening prior adaptive lasso (DSPAL) to select confounders and predictors of the outcome for multivariate treatments, which combines the adaptive lasso method with the marginal conditional (in)dependence prior information to select target covariates, in order to eliminate confounding bias and improve statistical efficiency. The distinctive features of our proposal are that it can be applied to high-dimensional or even ultra-high dimensional covariates for multivariate treatments, and can deal with the cases of both parametric and nonparametric outcome models, which makes it more robust compared to other methods. Our theoretical analyses show that the proposed procedure enjoys the sure screening property, the ranking consistency property and the variable selection consistency. Through a simulation study, we demonstrate that the proposed approach selects all confounders and predictors consistently and estimates the multivariate treatment effects with smaller bias and mean squared error compared to several alternatives under various scenarios. In real data analysis, the method is applied to estimate the causal effect of a three-dimensional continuous environmental treatment on cholesterol level and enlightening results are obtained.

Determinants of Cannabis Initiation and Cessation among Berlin College Students – A Longitudinal Study

Background: As a big European city famous for its party scene, Berlin attracts college students that are a high-risk population for cannabis use and use disorder. College years are often associated with new behavior patterns, but the factors leading to cannabis initiation are rarely studied past adolescence. This study describes the longitudinal evolution of college students’ cannabis use over two years and its correlates. Method: Data was collected among all students of Berlin’s public colleges via two online surveys (N = 1,201, mean interval = 16 months). Multivariable binary logistic regressions were performed on four outcomes: regular use, use initiation, use reduction and use cessation. Several dimensions of covariates were used: socio-demographic factors, psychological (locus of control, impulsivity, psychiatric diagnosis), behavioral (other substance use), perceived harm, declared intention to reduce and setting of cannabis use. Results: Overall, the majority of respondents did not change their cannabis use. The factors for use initiation (impulsivity, tobacco and alcohol use) were not fully symmetric to the factors leading to cutting down/quitting (locus of control, perceived harm, tobacco use). Perceived harm had an impact on quitting, but not on reducing use. The intention to reduce did not significantly predict subsequent use behavior. Most regular users use cannabis at home, which was associated with a low probability to reduce. Conclusions: No simple symmetry exists between correlates of initiation and cessation: tobacco co-use is important for both, while impulsivity and alcohol use lead to initiation and internal locus of control facilitates cessation.

Forecasting Value-at-Risk Using Deep Neural Network Quantile Regression

Abstract In this article, we use a deep quantile estimator, based on neural networks and their universal approximation property to examine a non-linear association between the conditional quantiles of a dependent variable and predictors. This methodology is versatile and allows both the use of different penalty functions, as well as high dimensional covariates. We present a Monte Carlo exercise where we examine the finite sample properties of the deep quantile estimator and show that it delivers good finite sample performance. We use the deep quantile estimator to forecast value-at-risk and find significant gains over linear quantile regression alternatives and other models, which are supported by various testing schemes. Further, we consider also an alternative architecture that allows the use of mixed frequency data in neural networks. This article also contributes to the interpretability of neural network output by making comparisons between the commonly used Shapley Additive Explanation values and an alternative method based on partial derivatives.

Model checking for parametric single-index models with massive datasets

Model checking is essential for making reliable statistical inferences. For massive datasets, we develop a new distributed method for testing parametric single-index models by integrating the divide and conquer strategy into the dimension reduction model-adaptive (DRMA) test. A distributed method for the determination of the structural dimension is also proposed. The asymptotic behaviors of the proposed test statistic under the null and alternative model are derived, which shows that the proposed test has the same limiting behavior of the DRMA test based on the entire dataset. In addition, the proposed test achieves adaptive rate-optimality using the sample-splitting strategy for selecting bandwidth. Our simulation results and a data illustration demonstrate that the proposed test performs better than the existing tests for massive datasets, especially when the dimension of covariates is large.

Asymptotics of the general GEE estimator for high-dimensional longitudinal data

In this study, we consider a class of high-dimensional longitudinal data with a large number of covariates, which arises frequently in the fields of bioinformatics and public health studies. In this setting, we study some asymptotic properties of the general generalized estimating equation (GEE) estimator when the dimension of covariates and the sample size reach infinity. Specifically, we establish the existence, consistency, and asymptotic normality of the GEE estimator under appropriate regularity conditions. The performance of the established theory is illustrated using Monte Carlo simulations. The main result of this study is a generalization of the results on natural link functions by considering nonnatural link functions.

Complete subset averaging approach for high-dimensional generalized linear models

This study proposes a novel complete subset averaging (CSA) method for high-dimensional generalized linear models based on a penalized Kullback–Leibler (KL) loss. All models under consideration can be potentially misspecified, and the dimension of covariates is allowed to diverge to infinity. The uniform convergence rate and asymptotic normality of the proposed estimator are established. Moreover, it is asymptotically optimal in terms of achieving the lowest KL loss. To ease the computational burden, we randomly draw a fixed number of subsets from the complete subsets and show their asymptotic equivalence. The Monte Carlo simulation and empirical application demonstrate that the proposed CSA method outperforms popular model-averaging methods.