Dimensional Covariance Research Articles

Improved model checking methods for parametric models with responses missing at random

In this paper, we consider the lack-of-fit test of a parametric model when the response variable is missing at random. The popular imputation and inverse probability weighting methods are first employed to tackle the missing data. Then by employing the projection technique, we propose empirical-process-based testing methods to check the appropriateness of the parametric model. The asymptotic properties of the test statistics are obtained under the null and local alternative hypothetical models. It is shown that the proposed testing methods are consistent, and can detect local alternative hypothetical models converging to the null model at the parametric rate. To determine the critical values, a consistent bootstrap method is proposed, and its asymptotic properties are established. The simulation results show that the tests outperform the existing methods in terms of empirical sizes and powers, especially under the situation with high dimensional covariates. Analysis of a diabetes data set of Pima Indians is carried out to demonstrate the application of the testing procedures.

Examining the specific dimensions of distress tolerance that prospectively predict perceived stress

We examined five dimensions of distress tolerance (i.e. uncertainty, ambiguity, frustration, negative emotion, physical discomfort) as prospective predictors of perceived stress. Undergraduate students (N = 135) completed self-report questionnaires over the course of two assessment sessions (T1 and T2). Results of a linear regression in which the five dimensions of distress tolerance and covariates (i.e. T1 perceived stress, duration between T1 and T2) served as predictor variables and T2 perceived stress served as the outcome variable showed that intolerance of uncertainty was the only dimension of distress tolerance to predict T2 perceived stress. To better understand this prospective association, we conducted a post hoc analysis simultaneously regressing two subdimensions of intolerance of uncertainty on T2 perceived stress. The subdimension representing beliefs that “uncertainty has negative behavioral and self-referent implications” significantly predicted T2 perceived stress, while the subdimension indicating that “uncertainty is unfair and spoils everything” did not. Results support a growing body of research suggesting intolerance of uncertainty as a risk factor for a wide variety of maladaptive psychological outcomes. Clinical implications will be discussed.

Generalized reduced rank latent factor regression for high dimensional tensor fields, and neuroimaging-genetic applications

We propose a generalized reduced rank latent factor regression model (GRRLF) for the analysis of tensor field responses and high dimensional covariates. The model is motivated by the need from imaging-genetic studies to identify genetic variants that are associated with brain imaging phenotypes, often in the form of high dimensional tensor fields. GRRLF identifies from the structure in the data the effective dimensionality of the data, and then jointly performs dimension reduction of the covariates, dynamic identification of latent factors, and nonparametric estimation of both covariate and latent response fields. After accounting for the latent and covariate effects, GRLLF performs a nonparametric test on the remaining factor of interest. GRRLF provides a better factorization of the signals compared with common solutions, and is less susceptible to overfitting because it exploits the effective dimensionality. The generality and the flexibility of GRRLF also allow various statistical models to be handled in a unified framework and solutions can be efficiently computed. Within the field of neuroimaging, it improves the sensitivity for weak signals and is a promising alternative to existing approaches. The operation of the framework is demonstrated with both synthetic datasets and a real-world neuroimaging example in which the effects of a set of genes on the structure of the brain at the voxel level were measured, and the results compared favorably with those from existing approaches.

Dimension reduction estimation for probability density with data missing at random when covariables are present

We develop dimension reduction estimating methods for probability density with data missing at random in the presence of covariables. In this paper, we propose two families of sufficient dimension reduction based nonparametric density estimators by modifying the regression calibration estimator and the inverse probability weighted estimator due to Wang (2008). The proposed methods overcome the challenges faced with high dimensional covariates: model specification and curse of dimensionality. The curse of dimensionality is overcome by replacing the covariables Xi in the regression calibration estimator and the inverse probability weighted estimator, respectively, with a root-n consistent estimator Sˆ(Xi) of a score S(Xi) for i=1,2,…,n. Three different scores S(⋅) are found by dimension reduction techniques. It is shown that the two families of proposed estimators are asymptotically normal, respectively, by taking three different scores. The asymptotic variances are the same when the same score is taken. With different scores, the asymptotic variances are different. A comparison for the two families of density estimators is made by taking different scores. Simulations are carried out to demonstrate the excellent performances of the proposed methods. A real data analysis is used to illustrate our methods.

Regularized LRT for large scale covariance matrices: One sample problem

The main theme of this paper is a modification of the likelihood ratio test (LRT) for testing high dimensional covariance matrix. Recently, the correct asymptotic distribution of the LRT for a large-dimensional case (the case p/n approaches to a constant γ∈(0,1]) is specified by researchers. The correct procedure is named as corrected LRT. Despite of its correction, the corrected LRT is a function of sample eigenvalues that are suffered from redundant variability from high dimensionality and, subsequently, still does not have full power in differentiating hypotheses on the covariance matrix. In this paper, motivated by the successes of a linearly shrunken covariance matrix estimator (simply shrinkage estimator) in various applications, we propose a regularized LRT that uses, in defining the LRT, the shrinkage estimator instead of the sample covariance matrix. We compute the asymptotic distribution of the regularized LRT, when the true covariance matrix is the identity matrix and a spiked covariance matrix. The obtained asymptotic results have applications in testing various hypotheses on the covariance matrix. Here, we apply them to testing the identity of the true covariance matrix, which is a long standing problem in the literature, and show that the regularized LRT outperforms the corrected LRT, which is its non-regularized counterpart. In addition, we compare the power of the regularized LRT to those of recent non-likelihood based procedures.

Model-free feature screening for ultrahigh dimensional censored regression

In this paper we design a sure independent ranking and screening procedure for censored regression (cSIRS, for short) with ultrahigh dimensional covariates. The inverse probability weighted cSIRS p...

Testing super-diagonal structure in high dimensional covariance matrices

The covariance matrices are essential quantities in econometric and statistical applications including portfolio allocation, asset pricing and factor analysis. Testing the entire covariance under high dimensionality endures large variability and causes a dilution of the signal-to-noise ratio and hence a reduction in the power. We consider a more powerful test procedure that focuses on testing along the super-diagonals of the high dimensional covariance matrix, which can infer more accurately on the structure of the covariance. We show that the test is powerful in detecting sparse signals and parametric structures in the covariance. The properties of the test are demonstrated by theoretical analyses, simulation and empirical studies.

Pooled shrinkage estimator for quadratic discriminant classifier: an analysis for small sample sizes in face recognition

The quadratic discriminant classifier (QDC) is a well-known parametric Bayesian classifier that has been successfully applied to statistical pattern recognition problems. One such application is in automatic face recognition where the number of training images per subject is often found to be much less than the length of the facial features. In such a case, the QDC cannot be used because the class-specific covariance matrix on which it depends is either poorly estimated or singular thereby resulting in unacceptable classifier performance. High dimensional covariance estimation techniques such as shrinkage can alleviate this problem but only to a certain extent. This paper presents a computationally simple yet effective solution for further improving the QDC performance in small sample size scenarios. The proposed technique adopts a strategy of combining the class-specific shrinkage estimates of the covariance matrix to obtain a pooled shrinkage estimate, which is then plugged into the QDC. Experiments indicate that the proposed classifier leads to remarkable improvement in face recognition accuracy as compared to the existing classifiers such as the nearest neighbor, support vector machine and naive Bayes, irrespective of the nature of the database and feature extraction method. Monte Carlo simulations reveal that this improvement is due to the much lower mean squared error of the pooled shrinkage estimator which offers greater stability to the QDC.

Estimating Effects with Rare Outcomes and High Dimensional Covariates: Knowledge is Power.

Many of the secondary outcomes in observational studies and randomized trials are rare. Methods for estimating causal effects and associations with rare outcomes, however, are limited, and this represents a missed opportunity for investigation. In this article, we construct a new targeted minimum loss-based estimator (TMLE) for the effect or association of an exposure on a rare outcome. We focus on the causal risk difference and statistical models incorporating bounds on the conditional mean of the outcome, given the exposure and measured confounders. By construction, the proposed estimator constrains the predicted outcomes to respect this model knowledge. Theoretically, this bounding provides stability and power to estimate the exposure effect. In finite sample simulations, the proposed estimator performed as well, if not better, than alternative estimators, including a propensity score matching estimator, inverse probability of treatment weighted (IPTW) estimator, augmented-IPTW and the standard TMLE algorithm. The new estimator yielded consistent estimates if either the conditional mean outcome or the propensity score was consistently estimated. As a substitution estimator, TMLE guaranteed the point estimates were within the parameter range. We applied the estimator to investigate the association between permissive neighborhood drunkenness norms and alcohol use disorder. Our results highlight the potential for double robust, semiparametric efficient estimation with rare events and high dimensional covariates.

More powerful tests for sparse high-dimensional covariances matrices

This paper considers improving the power of tests for the identity and sphericity hypotheses regarding high dimensional covariance matrices. The power improvement is achieved by employing the banding estimator for the covariance matrices, which leads to significant reduction in the variance of the test statistics in high dimension. Theoretical justification and simulation experiments are provided to ensure the validity of the proposed tests. The tests are used to analyze a dataset from an acute lymphoblastic leukemia gene expression study for an illustration.

Covariance Matrix Estimation Via Network Structure

In this article, we employ a regression formulation to estimate the high dimensional covariance matrix for a given network structure. Using prior information contained in the network relationships, we model the covariance as a polynomial function of the symmetric adjacency matrix. Accordingly, the problem of estimating a high dimensional covariance matrix is converted to one of estimating low dimensional coefficients of the polynomial regression function, which we can accomplish using ordinary least squares or maximum likelihood. The resulting covariance matrix estimator based on the maximum likelihood approach is guaranteed to be positive definite even in finite samples. Under mild conditions, we obtain the theoretical properties of the resulting estimators. A Bayesian information criterion is also developed to select the order of the polynomial function. Simulation studies and empirical examples illustrate the usefulness of the proposed methods.

Partially linear additive quantile regression in ultra-high dimension

We consider a flexible semiparametric quantile regression model for analyzing high dimensional heterogeneous data. This model has several appealing features: (1) By considering different conditional quantiles, we may obtain a more complete picture of the conditional distribution of a response variable given high dimensional covariates. (2) The sparsity level is allowed to be different at different quantile levels. (3) The partially linear additive structure accommodates nonlinearity and circumvents the curse of dimensionality. (4) It is naturally robust to heavy-tailed distributions. In this paper, we approximate the nonlinear components using B-spline basis functions. We first study estimation under this model when the nonzero components are known in advance and the number of covariates in the linear part diverges. We then investigate a nonconvex penalized estimator for simultaneous variable selection and estimation. We derive its oracle property for a general class of nonconvex penalty functions in the presence of ultra-high dimensional covariates under relaxed conditions. To tackle the challenges of nonsmooth loss function, nonconvex penalty function and the presence of nonlinear components, we combine a recently developed convex-differencing method with modern empirical process techniques. Monte Carlo simulations and an application to a microarray study demonstrate the effectiveness of the proposed method. We also discuss how the method for a single quantile of interest can be extended to simultaneous variable selection and estimation at multiple quantiles.

Regularized Quantile Regression and Robust Feature Screening for Single Index Models.

We propose both a penalized quantile regression and an independence screening procedure to identify important covariates and to exclude unimportant ones for a general class of ultrahigh dimensional single-index models, in which the conditional distribution of the response depends on the covariates via a single-index structure. We observe that the linear quantile regression yields a consistent estimator of the direction of the index parameter in the single-index model. Such an observation dramatically reduces computational complexity in selecting important covariates in the single-index model. We establish an oracle property for the penalized quantile regression estimator when the covariate dimension increases at an exponential rate of the sample size. From a practical perspective, however, when the covariate dimension is extremely large, the penalized quantile regression may suffer from at least two drawbacks: computational expediency and algorithmic stability. To address these issues, we propose an independence screening procedure which is robust to model misspecification, and has reliable performance when the distribution of the response variable is heavily tailed or response realizations contain extreme values. The new independence screening procedure offers a useful complement to the penalized quantile regression since it helps to reduce the covariate dimension from ultrahigh dimensionality to a moderate scale. Based on the reduced model, the penalized linear quantile regression further refines selection of important covariates at different quantile levels. We examine the finite sample performance of the newly proposed procedure by Monte Carlo simulations and demonstrate the proposed methodology by an empirical analysis of a real data set.

Feature Screening in Ultrahigh Dimensional Cox's Model.

Survival data with ultrahigh dimensional covariates such as genetic markers have been collected in medical studies and other fields. In this work, we propose a feature screening procedure for the Cox model with ultrahigh dimensional covariates. The proposed procedure is distinguished from the existing sure independence screening (SIS) procedures (Fan, Feng and Wu, 2010, Zhao and Li, 2012) in that the proposed procedure is based on joint likelihood of potential active predictors, and therefore is not a marginal screening procedure. The proposed procedure can effectively identify active predictors that are jointly dependent but marginally independent of the response without performing an iterative procedure. We develop a computationally effective algorithm to carry out the proposed procedure and establish the ascent property of the proposed algorithm. We further prove that the proposed procedure possesses the sure screening property. That is, with the probability tending to one, the selected variable set includes the actual active predictors. We conduct Monte Carlo simulation to evaluate the finite sample performance of the proposed procedure and further compare the proposed procedure and existing SIS procedures. The proposed methodology is also demonstrated through an empirical analysis of a real data example.

Covariate dimension reduction for survival data via the Gaussian process latent variable model.

The analysis of high-dimensional survival data is challenging, primarily owing to the problem of overfitting, which occurs when spurious relationships are inferred from data that subsequently fail to exist in test data. Here, we propose a novel method of extracting a low-dimensional representation of covariates in survival data by combining the popular Gaussian process latent variable model with a Weibull proportional hazards model. The combined model offers a flexible non-linear probabilistic method of detecting and extracting any intrinsic low-dimensional structure from high-dimensional data. By reducing the covariate dimension, we aim to diminish the risk of overfitting and increase the robustness and accuracy with which we infer relationships between covariates and survival outcomes. In addition, we can simultaneously combine information from multiple data sources by expressing multiple datasets in terms of the same low-dimensional space. We present results from several simulation studies that illustrate a reduction in overfitting and an increase in predictive performance, as well as successful detection of intrinsic dimensionality. We provide evidence that it is advantageous to combine dimensionality reduction with survival outcomes rather than performing unsupervised dimensionality reduction on its own. Finally, we use our model to analyse experimental gene expression data and detect and extract a low-dimensional representation that allows us to distinguish high-risk and low-risk groups with superior accuracy compared with doing regression on the original high-dimensional data.

Elastic Net Constrained Stereotype Logit Model for Ordered Categorical Data.

Gene expression studies are of growing importance in the field of medicine. In fact, sub-types within the same disease have been shown to have differing gene expression profiles. Often, researchers are interested in differentiating a disease by a categorical classification indicative of disease progression. For example, it may be of interest to identify genes that are associated with progression and to accurately predict the state of progression using gene expression data. One challenge when modeling microarray gene expression data is that there are more genes (variables) than there are observations. In addition, the genes usually demonstrate a complex variance-covariance structure. Therefore, modeling a categorical variable reflecting disease progression using gene expression data presents the need for methods capable of handling an ordinal outcome in the presence of a high dimensional covariate space. We present a method that combines the stereotype regression model with an elastic net penalty as a method capable of modeling an ordinal outcome for high-throughput genomic data sets. Results from the application of the proposed method to gene expression data are reported and the effectiveness of the proposed method is discussed.

Nonparametric tests of conditional treatment effects with an application to single-sex schooling on academic achievements

Summary We develop a general class of nonparametric tests for treatment effects conditional on covariates. We consider a wide spectrum of null hypotheses regarding conditional treatment effects, including the following: (a) the null hypothesis of the conditional stochastic dominance between treatment and control groups; (b) the null hypothesis that the conditional average treatment effect is nonpositive for each value of covariates; (c) the null hypothesis of no distributional (or average) treatment effect conditional on covariates. The test statistics are based on L1-type functionals of uniformly consistent nonparametric kernel estimators of conditional expectations that characterize the null hypotheses. We show that our tests using the standard normal critical values have asymptotically correct size. We also show that the proposed nonparametric tests are consistent against general fixed alternatives and have non-negligible powers against some n−1/2 local alternatives to the null hypothesis with inequality constraints and n−1/2h−d/4 local alternatives to the null hypothesis with equality constraints, where h is a bandwidth, n is the sample size and d is the dimension of continuous covariates. We illustrate the usefulness of our tests by applying them to the effect of single-sex schooling on academic achievements using Korean data.

Goodness-of-fit tests in semiparametric transformation models

Consider a semiparametric transformation model of the form \(\varLambda _{\theta }(Y)\) \(= m(X) + \varepsilon \), where \(Y\) is a univariate dependent variable, \(X\) is a \(d\)-dimensional covariate, and \(\varepsilon \) is independent of \(X\) and has mean zero. We assume that \(\{ \varLambda _{\theta } : \theta \in \varTheta \}\) is a parametric family of strictly increasing functions, while \(m\) is an unknown regression function. The goal of the paper is to develop tests for the null hypothesis that \(m(\cdot )\) belongs to a certain parametric family of regression functions. We propose a Kolmogorov–Smirnov and a Cramér–von Mises type test statistic, which measure the distance between the distribution of \(\varepsilon \) estimated under the null hypothesis and the distribution of \(\varepsilon \) without making use of this null hypothesis. The estimated distributions are based on a profile likelihood estimator of \(\theta \) and a local polynomial estimator of \(m(\cdot )\). The limiting distributions of these two test statistics are established under the null hypothesis and under a local alternative. We use a bootstrap procedure to approximate the critical values of the test statistics under the null hypothesis. Finally, a simulation study is carried out to illustrate the performance of our testing procedures, and we apply our tests to data on the scattering of sunlight in the atmosphere.

A new test for part of high dimensional regression coefficients

It is well known that the F-test breaks down completely when the dimension of covariates exceeds the sample size. This paper proposes a new test for part of regression coefficients in high dimensional linear models. Under the high dimensional null hypothesis and various scenarios of the alternative, we derive the asymptotic distribution of the proposed test statistic, which allows power evaluation of the test. Through simulation studies, we demonstrate good finite-sample performance of the proposed test in comparison with the existing methods. The practical utility of our method is illustrated by a real data example.

High dimensional covariance matrix estimation using multi-factor models from incomplete information

Covariance matrix plays an important role in risk management, asset pricing, and portfolio allocation. Covariance matrix estimation becomes challenging when the dimensionality is comparable or much larger than the sample size. A widely used approach for reducing dimensionality is based on multi-factor models. Although it has been well studied and quite successful in many applications, the quality of the estimated covariance matrix is often degraded due to a nontrivial amount of missing data in the factor matrix for both technical and cost reasons. Since the factor matrix is only approximately low rank or even has full rank, existing matrix completion algorithms are not applicable. We consider a new matrix completion paradigm using the factor models directly and apply the alternating direction method of multipliers for the recovery. Numerical experiments show that the nuclear-norm matrix completion approaches are not suitable but our proposed models and algorithms are promising.