Dimensional Covariance Research Articles

FEATURE SELECTION FOR GENERALIZED VARYING COEFFICIENT MIXED-EFFECT MODELS WITH APPLICATION TO OBESITY GWAS.

Motivated by an empirical analysis of data from a genome-wide association study on obesity, measured by the body mass index (BMI), we propose a two-step gene-detection procedure for generalized varying coefficient mixed-effects models with ultrahigh dimensional covariates. The proposed procedure selects significant single nucleotide polymorphisms (SNPs) impacting the mean BMI trend, some of which have already been biologically proven to be "fat genes." The method also discovers SNPs that significantly influence the age-dependent variability of BMI. The proposed procedure takes into account individual variations of genetic effects and can also be directly applied to longitudinal data with continuous, binary or count responses. We employ Monte Carlo simulation studies to assess the performance of the proposed method and further carry out causal inference for the selected SNPs.

Variable selection and estimation for high‐dimensional spatial autoregressive models

AbstractSpatial regression models are important tools for many scientific disciplines including economics, business, and social science. In this article, we investigate postmodel selection estimators that apply least squares estimation to the model selected by penalized estimation in high‐dimensional regression models with spatial autoregressive errors. We show that by separating the model selection and estimation process, the postmodel selection estimator performs at least as well as the simultaneous variable selection and estimation method in terms of the rate of convergence. Moreover, under perfect model selection, the ℓ2 rate of convergence is the oracle rate of , compared with the convergence rate of in the general case. Here, n is the sample size and p, s are the model dimension and number of significant covariates, respectively. We further provide the convergence rate of the estimation error in the form of norm, and ideally the rate can reach as fast as .

Iterative reweighted proximal projection based DOA estimation algorithm for monostatic MIMO radar

The problem of nonconvex and nonsmooth sparse representation for direction of arrival (DOA) estimation in monostatic multiple-input multiple-output (MIMO) radar is addressed in this paper, which is dealt with by a novel iterative reweighted proximal projection method. The proposed method firstly obtains the array covariance vector by performing the vectorization operation on the reduced dimensional covariance matrix. Then a sparse representation framework is formulated for DOA estimation through minimizing the nonconvex and nonsmooth sparsity promoting function, and the weighted matrix, which is based on the high-order power of the inverse of the reduced dimensional covariance matrix, is designed for reweighting the nonconvex and nonsmooth minimization to enhance the sparsity of the solution. Thereafter, an iterative algorithm using proximal projection approach along with reweighted penalty ideas is developed to recover the sparse solution. Finally, DOA estimation is accomplished by searching the spectrum of the solution. Due to achieving a better approximation to the l0 norm, the proposed method exhibits better DOA estimation accuracy than the reweighted l1-SVD algorithm and reweighted SL0 algorithm. Furthermore, the proposed method can avoid any a-priori information on the number of targets. Simulation results are presented to verify the effectiveness of the proposed method.

Nonparametric Estimation of Conditional Expectation with Auxiliary Information and Dimension Reduction

Nonparametric estimation of the conditional expectation of an outcome Y given a covariate vector U is of primary importance in many statistical applications such as prediction and personalized medicine. In some problems, there is an additional auxiliary variable Z in the training dataset used to construct estimators, but Z is not available for future prediction or selecting patient treatment in personalized medicine. For example, in the training dataset longitudinal outcomes are observed, but only the last outcome Y is concerned in the future prediction or analysis. The longitudinal outcomes other than the last point is then the variable Z that is observed and related with both Y and U. Previous work on how to make use of Z in the estimation of mainly focused on using Z in the construction of a linear function of U to reduce covariate dimension for better estimation. Using , we propose a two-step estimation of inner and outer expectations, respectively, with sufficient dimension reduction for kernel estimation in both steps. The information from Z is utilized not only in dimension reduction, but also directly in the estimation. Because of the existence of different ways for dimension reduction, we construct two estimators that may improve the estimator without using Z. The improvements are shown in the convergence rate of estimators as the sample size increases to infinity as well as in the finite sample simulation performance. A real data analysis about the selection of mammography intervention is presented for illustration.

Right Singular Vector Projection Graphs: Fast High Dimensional Covariance Matrix Estimation under Latent Confounding

SummaryWe consider the problem of estimating a high dimensional p × p covariance matrix Σ, given n observations of confounded data with covariance Σ+ΓΓT, where Γ is an unknown p × q matrix of latent factor loadings. We propose a simple and scalable estimator based on the projection onto the right singular vectors of the observed data matrix, which we call right singular vector projection (RSVP). Our theoretical analysis of this method reveals that, in contrast with approaches based on the removal of principal components, RSVP can cope well with settings where the smallest eigenvalue of ΓTΓ is relatively close to the largest eigenvalue of Σ, as well as when the eigenvalues of ΓTΓ are diverging fast. RSVP does not require knowledge or estimation of the number of latent factors q, but it recovers Σ only up to an unknown positive scale factor. We argue that this suffices in many applications, e.g. if an estimate of the correlation matrix is desired. We also show that, by using subsampling, we can further improve the performance of the method. We demonstrate the favourable performance of RSVP through simulation experiments and an analysis of gene expression data sets collated by the GTEX consortium.

Estimation of a semiparametric transformation model: A novel approach based on least squares minimization

Consider the following semiparametric transformation model $\Lambda_{\theta }(Y)=m(X)+\varepsilon $, where $X$ is a $d$-dimensional covariate, $Y$ is a univariate response variable and $\varepsilon $ is an error term with zero mean and independent of $X$. We assume that $m$ is an unknown regression function and that $\{\Lambda _{\theta }:\theta \in\Theta \}$ is a parametric family of strictly increasing functions. Our goal is to develop two new estimators of the transformation parameter $\theta $. The main idea of these two estimators is to minimize, with respect to $\theta $, the $L_{2}$-distance between the transformation $\Lambda _{\theta }$ and one of its fully nonparametric estimators. We consider in particular the nonparametric estimator based on the least-absolute deviation loss constructed in Colling and Van Keilegom (2019). We establish the consistency and the asymptotic normality of the two proposed estimators of $\theta $. We also carry out a simulation study to illustrate and compare the performance of our new parametric estimators to that of the profile likelihood estimator constructed in Linton et al. (2008).

Feature Screening in Ultrahigh Dimensional Generalized Varying-coefficient Models.

Generalized varying coefficient models are particularly useful for examining dynamic effects of covariates on a continuous, binary or count response. This paper is concerned with feature screening for generalized varying coefficient models with ultrahigh dimensional covariates. The proposed screening procedure is based on joint quasi-likelihood of all predictors, and therefore is distinguished from marginal screening procedures proposed in the literature. In particular, the proposed procedure can effectively identify active predictors that are jointly dependent but marginally independent of the response. In order to carry out the proposed procedure, we propose an effective algorithm and establish the ascent property of the proposed algorithm. We further prove that the proposed procedure possesses the sure screening property. That is, with probability tending to one, the selected variable set includes the actual active predictors. We examine the finite sample performance of the proposed procedure and compare it with existing ones via Monte Carlo simulations, and illustrate the proposed procedure by a real data example.

Regression with I-priors

The problem of estimating a parametric or nonparametric regression function in a model with normal errors is considered. For this purpose, a novel objective prior for the regression function is proposed, defined as the distribution maximizing entropy subject to a suitable constraint based on the Fisher information on the regression function. The prior is named I-prior. For the present model, it is Gaussian with covariance kernel proportional to the Fisher information, and mean chosen a priori (e.g., 0).The I-prior has the intuitively appealing property that the more information is available about a linear functional of the regression function, the larger its prior variance, and, broadly speaking, the less influential the prior is on the posterior. Unlike the Jeffreys prior, it can be used in high dimensional settings. The I-prior methodology can be used as a principled alternative to Tikhonov regularization, which suffers from well-known theoretical problems which are briefly reviewed.The regression function is assumed to lie in a reproducing kernel Hilbert space (RKHS) over a low or high dimensional covariate space, giving a high degree of generality. Analysis of some real data sets and a small-scale simulation study show competitive performance of the I-prior methodology, which is implemented in the R-package iprior.

Estimation of fully nonparametric transformation models

Consider the following nonparametric transformation model $\Lambda (Y)=m(X)+\varepsilon $, where $X$ is a $d$-dimensional covariate, $Y$ is a continuous univariate dependent variable and $\varepsilon $ is an error term with zero mean and which is independent of $X$. We assume that the unknown transformation $\Lambda $ is strictly increasing and that $m$ is an unknown regression function. Our goal is to develop two new nonparametric estimators of the transformation $\Lambda $, the first one based on the least squares loss and the second one based on the least absolute deviation loss, and to compare their performance with that of the estimators developed by Chiappori, Komunjer and Kristensen (J. Econometrics 188 (2015) 22–39). Our proposed estimators are based on an estimator of the conditional distribution of $U$ given $X$, where $U$ is an appropriate transformation of $Y$ that is uniformly distributed. The main motivation for working with $U$ instead of $Y$ is that, in transformation models, the response $Y$ is often skewed with very long tails, and so kernel smoothing based on $Y$ does not work well. Hence, we expect to obtain better estimators if we pre-transform $Y$ before applying kernel smoothing. We establish the asymptotic normality of the two proposed estimators. We also carry out a simulation study to illustrate the performance of our estimators, to compare these new estimators with the ones of Chiappori, Komunjer and Kristensen (J. Econometrics 188 (2015) 22–39) and to see under which model conditions which estimators behave the best.

Least squares estimation in the monotone single index model

We study the monotone single index model where a real response variable $Y$ is linked to a $d$-dimensional covariate $X$ through the relationship $E[Y|X]=\Psi_{0}(\alpha^{T}_{0}X)$, almost surely. Both the ridge function, $\Psi_{0}$, and the index parameter, $\alpha_{0}$, are unknown and the ridge function is assumed to be monotone. Under some appropriate conditions, we show that the rate of convergence in the $L_{2}$-norm for the least squares estimator of the bundled function $\Psi_{0}({\alpha}^{T}_{0}\cdot)$ is $n^{1/3}$. A similar result is established for the isolated ridge function, and the index is shown to converge at least at the rate $n^{1/3}$. Since the least squares estimator of the index is computationally intensive, we also consider alternative estimators of the index $\alpha_{0}$ from earlier literature. Moreover, we show that if the rate of convergence of such an alternative estimator is at least $n^{1/3}$, then the corresponding least-squares type estimators (obtained via a “plug-in” approach) of both the bundled and isolated ridge functions still converge at the rate $n^{1/3}$.

HYPOTHESIS TESTING ON LINEAR STRUCTURES OF HIGH DIMENSIONAL COVARIANCE MATRIX.

This paper is concerned with test of significance on high dimensional covariance structures, and aims to develop a unified framework for testing commonly-used linear covariance structures. We first construct a consistent estimator for parameters involved in the linear covariance structure, and then develop two tests for the linear covariance structures based on entropy loss and quadratic loss used for covariance matrix estimation. To study the asymptotic properties of the proposed tests, we study related high dimensional random matrix theory, and establish several highly useful asymptotic results. With the aid of these asymptotic results, we derive the limiting distributions of these two tests under the null and alternative hypotheses. We further show that the quadratic loss based test is asymptotically unbiased. We conduct Monte Carlo simulation study to examine the finite sample performance of the two tests. Our simulation results show that the limiting null distributions approximate their null distributions quite well, and the corresponding asymptotic critical values keep Type I error rate very well. Our numerical comparison implies that the proposed tests outperform existing ones in terms of controlling Type I error rate and power. Our simulation indicates that the test based on quadratic loss seems to have better power than the test based on entropy loss.

A fast divide-and-conquer sparse Cox regression.

We propose a computationally and statistically efficient divide-and-conquer (DAC) algorithm to fit sparse Cox regression to massive datasets where the sample size $n_0$ is exceedingly large and the covariate dimension $p$ is not small but $n_0\gg p$. The proposed algorithm achieves computational efficiency through a one-step linear approximation followed by a least square approximation to the partial likelihood (PL). These sequences of linearization enable us to maximize the PL with only a small subset and perform penalized estimation via a fast approximation to the PL. The algorithm is applicable for the analysis of both time-independent and time-dependent survival data. Simulations suggest that the proposed DAC algorithm substantially outperforms the full sample-based estimators and the existing DAC algorithm with respect to the computational speed, while it achieves similar statistical efficiency as the full sample-based estimators. The proposed algorithm was applied to extraordinarily large survival datasets for the prediction of heart failure-specific readmission within 30 days among Medicare heart failure patients.

Robust inference for estimating equations with nonignorably missing data based on SIR algorithm

ABSTRACTNonresponse is a very common phenomenon in survey sampling. Nonignorable nonresponse – that is, a response mechanism that depends on the values of the variable having nonresponse – is the most difficult type of nonresponse to handle. This article develops a robust estimation approach to estimating equations (EEs) by incorporating the modelling of nonignorably missing data, the generalized method of moments (GMM) method and the imputation of EEs via the observed data rather than the imputed missing values when some responses are subject to nonignorably missingness. Based on a particular semiparametric logistic model for nonignorable missing response, this paper proposes the modified EEs to calculate the conditional expectation under nonignorably missing data. We can apply the GMM to infer the parameters. The advantage of our method is that it replaces the non-parametric kernel-smoothing with a parametric sampling importance resampling (SIR) procedure to avoid nonparametric kernel-smoothing problems with high dimensional covariates. The proposed method is shown to be more robust than some current approaches by the simulations.

Feature screening for ultrahigh-dimensional survival data when failure indicators are missing at random

In modern statistical applications, the dimension of covariates can be much larger than the sample size, and extensive research has been done on screening methods which can effectively reduce the dimensionality. However, the existing feature screening procedure can not be used to handle the ultrahigh-dimensional survival data problems when failure indicators are missing at random. This motivates us to develop a feature screening procedure to handle this case. In this paper, we propose a feature screening procedure by sieved nonparametric maximum likelihood technique for ultrahigh-dimensional survival data with failure indicators missing at random. The proposed method has several desirable advantages. First, it does not rely on any model assumption and works well for nonlinear survival regression models. Second, it can be used to handle the incomplete survival data with failure indicators missing at random. Third, the proposed method is invariant under the monotone transformation of the response and satisfies the sure screening property. Simulation studies are conducted to examine the performance of our approach, and a real data example is also presented for illustration.

A new joint screening method for right-censored time-to-event data with ultra-high dimensional covariates.

In an ultra-high dimensional setting with a huge number of covariates, variable screening is useful for dimension reduction before applying a more refined method for model selection and statistical analysis. This paper proposes a new sure joint screening procedure for right-censored time-to-event data based on a sparsity-restricted semiparametric accelerated failure time model. Our method, referred to as Buckley-James assisted sure screening (BJASS), consists of an initial screening step using a sparsity-restricted least-squares estimate based on a synthetic time variable and a refinement screening step using a sparsity-restricted least-squares estimate with the Buckley-James imputed event times. The refinement step may be repeated several times to obtain more stable results. We show that with any fixed number of refinement steps, the BJASS procedure retains all important variables with probability tending to 1. Simulation results are presented to illustrate its performance in comparison with some marginal screening methods. Real data examples are provided using a diffuse large-B-cell lymphoma (DLBCL) data and a breast cancer data. We have implemented the BJASS method using Matlab and made it available to readers through Github https://github.com/yiucla/BJASS .

Prediction with High Dimensional Regression Via Hierarchically Structured Gaussian Mixtures and Latent Variables

Summary We propose a hierarchical Gaussian locally linear mapping structured mixture model, named HGLLiM, to predict low dimensional responses based on high dimensional covariates when the associations between the responses and the covariates are non-linear. For tractability, HGLLiM adopts inverse regression to handle the high dimension and locally linear mappings to capture potentially non-linear relations. Data with similar associations are grouped together to form a cluster. A mixture is composed of several clusters following a hierarchical structure. This structure enables shared covariance matrices and latent factors across smaller clusters to limit the number of parameters to estimate. Moreover, HGLLiM adopts a robust estimation procedure for model stability. We use three real data sets to demonstrate different features of HGLLiM. With the face data set, HGLLiM shows ability to model non-linear relationships through mixtures. With the orange juice data set, we show that the prediction performance of HGLLiM is robust to the presence of outliers. Moreover, we demonstrate that HGLLiM is capable of handling large-scale complex data by using the data acquired from a magnetic resonance vascular fingerprinting study. These examples illustrate the wide applicability of HGLLiM to handle different aspects of a complex data structure in prediction.

Nonnegative estimation and variable selection via adaptive elastic-net for high-dimensional data

This paper proposes the nonnegative adaptive elastic-net for simultaneous nonnegative estimation and variable selection in sparse high-dimensional linear regression models. By inheriting the good features of adaptive elastic-net, the nonnegative adaptive elastic-net enjoys the oracle property even in high-dimensional settings where the dimension of covariates can be much larger than the sample size. Through the simulation, we show that the newly proposed method deals with the collinearity problem better than alternative procedures in the literature. To make the proposed method practically feasible, we extend the multiplicative updates algorithm for implementation. Finally, we illustrate the favorable finite-sample performance of the proposed method through tracking the CSI 300 index, an important stock market index in China.

A Sparse Random Projection-Based Test for Overall Qualitative Treatment Effects

In contrast to the classical “one-size-fits-all” approach, precision medicine proposes the customization of individualized treatment regimes to account for patients’ heterogeneity in response to treatments. Most of existing works in the literature focused on estimating optimal individualized treatment regimes. However, there has been less attention devoted to hypothesis testing regarding the existence of overall qualitative treatment effects, especially when there are a large number of prognostic covariates. When covariates do not have qualitative treatment effects, the optimal treatment regime will assign the same treatment to all patients regardless of their covariate values. In this article, we consider testing the overall qualitative treatment effects of patients’ prognostic covariates in a high-dimensional setting. We propose a sample splitting method to construct the test statistic, based on a nonparametric estimator of the contrast function. When the dimension of covariates is large, we construct the test based on sparse random projections of covariates into a low-dimensional space. We prove the consistency of our test statistic. In the regular cases, we show the asymptotic power function of our test statistic is asymptotically the same as the “oracle” test statistic which is constructed based on the “optimal” projection matrix. Simulation studies and real data applications validate our theoretical findings. Supplementary materials for this article are available online.

Sparse linear regression models of high dimensional covariates with non-Gaussian outliers and Berkson error-in-variable under heteroscedasticity

The use of penalized least squares based estimations in high dimensional linear regression has been received a great deal of attention during past decades and applied to variable selections under sparsity scenarios. However, it remains unclear that complex data with multiple uncertainties in measurements, such as outliers, error-in-variables and heteroscedasticity, to what extent, can influence these approaches. This paper, here, demonstrates the behaviors of least-squares based approaches in complex data problems, and suggests an alternative approach, Wald-type estimator, relying on theoretical perspectives. Numerical experiments and real data analysis are also presented to illustrate the performance of the different estimation strategies.

Robust Model Selection and Estimation for Censored Survival Data with High Dimensional Genomic Covariates.

When relating genomic data to survival outcomes, there are three main challenges that are the censored survival outcomes, the high-dimensionality of the genomic data, and the non-normality of data. We propose a method to tackle these challenges simultaneously and obtain a robust estimation of detecting significant genes related to survival outcomes based on Accelerated Failure Time (AFT) model. Specifically, we include a general loss function to the AFT model, adopt model regularization and shrinkage technique, cope with parameters tuning and model selection, and develop an algorithm based on unified Expectation-Maximization approach for easy implementation. Simulation results demonstrate the advantages of the proposed method compared with existing methods when the data has heavy-tailed errors and correlated covariates. Two real case studies on patients are provided to illustrate the application of the proposed method.