High-dimensional Setups Research Articles

Estimating Haplotype Structure and Frequencies: A Bayesian Approach to Unknown Design in Pooled Genomic Data.

The estimation of haplotype structure and frequencies provides crucial information about the composition of genomes. Techniques, such as single-individual haplotyping, aim to reconstruct individual haplotypes from diploid genome sequencing data. However, our focus is distinct. We address the challenge of reconstructing haplotype structure and frequencies from pooled sequencing samples where multiple individuals are sequenced simultaneously. A frequentist method to address this issue has recently been proposed. In contrast to this and other methods that compute point estimates, our proposed Bayesian hierarchical model delivers a posterior that permits us to also quantify uncertainty. Since matching permutations in both haplotype structure and corresponding frequency matrix lead to the same reconstruction of their product, we introduce an order-preserving shrinkage prior that ensures identifiability with respect to permutations. For inference, we introduce a blocked Gibbs sampler that enforces the required constraints. In a simulation study, we assessed the performance of our method. Furthermore, by using our approach on two distinct sets of real data, we demonstrate that our Bayesian approach can reconstruct the dominant haplotypes in a challenging, high-dimensional set-up.

Is the empirical out-of-sample variance an informative risk measure for the high-dimensional portfolios?

The main contribution of this paper is the derivation of the asymptotic behavior of the out-of-sample variance, the out-of-sample relative loss, and of their empirical counterparts in the high-dimensional setting, i.e., when both ratios p/n and p/m tend to some positive constants as m→∞ and n→∞, where p is the portfolio dimension, while n and m are the sample sizes from the in-sample and out-of-sample periods, respectively. The results are obtained for the traditional estimator of the global minimum variance (GMV) portfolio and for the two shrinkage estimators introduced by Frahm and Memmel (2010) and Bodnar et al. (2018). We show that the behavior of the empirical out-of-sample variance may be misleading in many practical situations, leading, for example, to a comparison of zeros. On the other hand, this will never happen with the empirical out-of-sample relative loss, which seems to provide a natural normalization of the out-of-sample variance in the high-dimensional setup. As a result, an important question arises if the out-of-sample variance can safely be used in practice for portfolios constructed from a large asset universe.

Independence tests with random subspace of two random vectors in high dimension

Testing for independence between two random vectors is a fundamental problem in statistics. When the dimension of these two random vectors are fixed, the existing tests based on the distance covariance and Hilbert–Schmidt independence criterion with many desirable properties, including the capacity to capture linear and non-linear dependence. However, these tests may fail to capture the non-linear dependence due to the “curse of dimensionality” when the random vectors are high dimensional. To attack this problem, we propose a general framework for testing the dependence of two random vectors to randomly select two subspaces consisting of components of the vectors, respectively. To enhance the performance of this method, we repeatedly implement this procedure to construct the final test statistic. The new method can also work for non-linear dependence detection in a high-dimensional setup. Theoretically, if the replication time tends to infinity to get the final statistic, we can avoid potential power loss caused by lousy subspaces. Therefore, the two proposed tests are consistent with general alternatives. The weak limit under the null hypothesis is normal; thus, determining critical value need not resort to resampling approximation. We demonstrate the finite-sample performance of the proposed test by using Monte Carlo simulations and the analysis for a real-data example.

An Orthogonal Matching Pursuit Variable Screening Algorithm for High-Dimensional Linear Regression Models

Variable selection plays an important role in data mining. It is crucial to filter useful variables and extract useful information in a high-dimensional setup when the number of predictor variables d tends to be much larger than the sample size n . Statistical inferences can be more precise after irrelevant variables are moved out by the screening method. This article proposes an orthogonal matching pursuit algorithm for variable screening under the high-dimensional setup. The proposed orthogonal matching pursuit method demonstrates good performance in variable screening. In particular, if the dimension of the true model is finite, OMP might discover all relevant predictors within a finite number of steps. Throughout theoretical analysis and simulations, it is confirmed that the orthogonal matching pursuit algorithm can identify relevant predictors to ensure screening consistency in variable selection. Given the sure screening property, the BIC criterion can be used to practically select the best candidate from the models generated by the OMP algorithm. Compared with the traditional orthogonal matching pursuit method, the resulting model can improve prediction accuracy and reduce computational cost by screening out the relevant variables.

On Monte-Carlo methods in convex stochastic optimization

We develop a novel procedure for estimating the optimizer of general convex stochastic optimization problems of the form minx∈XE[F(x,ξ)], when the given data is a finite independent sample selected according to ξ. The procedure is based on a median-of-means tournament, and is the first procedure that exhibits the optimal statistical performance in heavy tailed situations: we recover the asymptotic rates dictated by the central limit theorem in a nonasymptotic manner once the sample size exceeds some explicitly computable threshold. Additionally, our results apply in the high-dimensional setup, as the threshold sample size exhibits the optimal dependence on the dimension (up to a logarithmic factor). The general setting allows us to recover recent results on multivariate mean estimation and linear regression in heavy-tailed situations and to prove the first sharp, nonasymptotic results for the portfolio optimization problem.

Bootstrap Tests for High-Dimensional White-Noise

The testing of white-noise (WN) is an essential step in time series analysis. In a high dimensional set-up, most existing methods either are computationally infeasible, or suffer from highly distorted Type-I errors, or both. We propose an easy-to-implement bootstrap method for high-dimensional WN test and prove its consistency for a variety of test statistics. Its power properties as well as extensions to WN tests based on fitted residuals are also considered. Simulation results show that compared to the existing methods, the new approach possesses much better power, while maintaining a proper control over the Type-I error. They also provide proofs that even in cases where our method is expected to suffer from lack of theoretical justification, it continues to outperform its competitors. The proposed method is applied to the analysis of the daily stock returns of the top 50 companies by market capitalization listed on the NYSE, and we find strong evidence that the common market factor is the main cause of cross-correlation between stocks.

Multivariate Overnight GARCH-Ito Model with Applications in Large Volatility Matrix Estimation and Prediction

This paper introduces a unified multivariate overnight GARCH-It\^o model for volatility matrix estimation and prediction both in the low- and high-dimensional set-up. To account for whole-day market dynamics in the financial market, the proposed model has two different instantaneous volatility processes for the open-to-close and close-to-open periods, while each embeds the discrete-time multivariate GARCH model structure. We call it the multivariate overnight GARCH-It\^o (MOGI) model. Based on the connection between the discrete-time model structure and the continuous-time diffusion process, we propose a weighted least squares estimation procedure for estimating model parameters with open-to-close high-frequency and close-to-open low-frequency financial data, and establish its asymptotic theorems. We further discuss the prediction of future vast volatility matrices and study its asymptotic properties. A simulation study is conducted to check the finite sample performance of the proposed estimation and prediction methods. The empirical analysis is carried out to compare the performance of the proposed MOGI model with other benchmark methods in portfolio allocation problems.

Identifying complex gene–gene interactions: a mixed kernel omnibus testing approach

Genes do not function independently; rather, they interact with each other to fulfill their joint tasks. Identification of gene-gene interactions has been critically important in elucidating the molecular mechanisms responsible for the variation of a phenotype. Regression models are commonly used to model the interaction between two genes with a linear product term. The interaction effect of two genes can be linear or nonlinear, depending on the true nature of the data. When nonlinear interactions exist, the linear interaction model may not be able to detect such interactions; hence, it suffers from substantial power loss. While the true interaction mechanism (linear or nonlinear) is generally unknown in practice, it is critical to develop statistical methods that can be flexible to capture the underlying interaction mechanism without assuming a specific model assumption. In this study, we develop a mixed kernel function which combines both linear and Gaussian kernels with different weights to capture the linear or nonlinear interaction of two genes. Instead of optimizing the weight function, we propose a grid search strategy and use a Cauchy transformation of the P-values obtained under different weights to aggregate the P-values. We further extend the two-gene interaction model to a high-dimensional setup using a de-biased LASSO algorithm. Extensive simulation studies are conducted to verify the performance of the proposed method. Application to two case studies further demonstrates the utility of the model. Our method provides a flexible and computationally efficient tool for disentangling complex gene-gene interactions associated with complex traits.

On some synchronization problems with multiple instances

Many classical synchronization problems such as the assembly line crew scheduling problem (ALCS), some data association problems or multisensor tracking problems can be formulated as finding intra-column rearrangements for a single matrix representing costs, distances, similarities or time requirements. In this paper, we consider an extension of these problems to the case of multiple matrices, reflecting various possible instances (scenarios). To approximate optimal rearrangements, we introduce the Block Swapping Algorithm (BSA) and a further customization of it that we call the customized Block Swapping Algorithm (Cust BSA). A numerical study shows that the two algorithms we propose – in particular Cust BSA – yield high-quality solutions and also deal efficiently with high-dimensional set-ups.

Iterative Variable Selection for High-Dimensional Data: Prediction of Pathological Response in Triple-Negative Breast Cancer

Over the last decade, regularized regression methods have offered alternatives for performing multi-marker analysis and feature selection in a whole genome context. The process of defining a list of genes that will characterize an expression profile remains unclear. It currently relies upon advanced statistics and can use an agnostic point of view or include some a priori knowledge, but overfitting remains a problem. This paper introduces a methodology to deal with the variable selection and model estimation problems in the high-dimensional set-up, which can be particularly useful in the whole genome context. Results are validated using simulated data and a real dataset from a triple-negative breast cancer study.

A new framework for distance and kernel-based metrics in high dimensions

The paper presents new metrics to quantify and test for (i) the equality of distributions and (ii) the independence between two high-dimensional random vectors. We show that the energy distance based on the usual Euclidean distance cannot completely characterize the homogeneity of two high-dimensional distributions in the sense that it only detects the equality of means and the traces of covariance matrices in the high-dimensional setup. We propose a new class of metrics which inherits the desirable properties of the energy distance and maximum mean discrepancy/(generalized) distance covariance and the Hilbert-Schmidt Independence Criterion in the low-dimensional setting and is capable of detecting the homogeneity of/completely characterizing independence between the low-dimensional marginal distributions in the high dimensional setup. We further propose t-tests based on the new metrics to perform high-dimensional two-sample testing/independence testing and study their asymptotic behavior under both high dimension low sample size (HDLSS) and high dimension medium sample size (HDMSS) setups. The computational complexity of the t-tests only grows linearly with the dimension and thus is scalable to very high dimensional data. We demonstrate the superior power behavior of the proposed tests for homogeneity of distributions and independence via both simulated and real datasets.

Asymmetric influence measure for high dimensional regression

Identification of influential observations is crucial in data analysis, particularly with high dimensional datasets, where the number of predictors is higher than the sample size. These rich datasets with extensive detail are increasingly exploited and analyzed in multiple fields of science, e.g., genomics, neuroscience, finance, etc. Unfortunately, classical diagnostic statistical tools are not tailored for identifying influential observations in high dimensional setup. In this paper, we use the concept of expectiles to develop an influence measure in high dimensional regression. The influence measure is based on the asymmetric marginal correlation, and its derived asymptotic distribution is used to define a threshold based on statistical principles. Our comprehensive simulation results display the favorable qualities of this influence measure under various scenarios. The usefulness of the proposed measure is illustrated through the analysis of a neuroimaging dataset. An R package implementing the procedure is publicly available on GitHub (https://github.com/AmBarry/hidetify).

On a Synchronization Problem With Multiple Instances

We study a synchronization problem with multiple instances. First, we show that the problem we consider can be formulated as the problem of finding an intra-column rearrangement for multiple matrices (which reflect problem instances) such that the row sums across the various matrices show minimum variability. To obtain the optimal rearrangement, we introduce the Block Swapping Algorithm (BSA) and a further customization of it that we label as the Customized Block Swapping Algorithm (Cust BSA). A numerical study shows that the two algorithms we propose yield high-quality solutions and also deal efficiently with high-dimensional set-ups.

Nonparametric Phase-II control charts for monitoring high-dimensional processes with unknown parameters

Monitoring multivariate and high-dimensional data streams is often an essential requirement for quality management in manufacturing and service sectors in the Industry 4.0 era. Identifying a suitable distribution for a multivariate data set, especially when the number of variables is much larger than the sample size, is often challenging. Consequently, in a high-dimensional set-up, that is, when the number of variables under investigation exceeds sample size, parametric methods are generally not reliable in practice. There are various nonparametric schemes based on data depth for multivariate process monitoring, which are applicable only when the sample size is reasonably larger than the number of variables in the process but not in a high-dimensional set-up. We discuss that most of these charts are not robust when the true process parameters are unknown. There are, however, some nonparametric schemes for a high-dimensional process, when true process parameters are known. Nevertheless, when process parameters are unknown, a highly robust nonparametric scheme for monitoring high-dimensional processes is not yet available. In this paper, we propose some Shewhart-type nonparametric monitoring schemes based on specific distance metrics for surveillance of multivariate as well as high-dimensional processes. Our proposed charts are easy to implement, interpret and also advantageous in simultaneous monitoring of multiple aspects. We discuss the design and implementation issues in details. We carry out a performance study using Monte Carlo simulations and illustrate the proposed methods using a dataset related to industrial production.

Spectral and matrix factorization methods for consistent community detection in multi-layer networks

We consider the problem of estimating a consensus community structure by combining information from multiple layers of a multi-layer network using methods based on the spectral clustering or a low-rank matrix factorization. As a general theme, these “intermediate fusion” methods involve obtaining a low column rank matrix by optimizing an objective function and then using the columns of the matrix for clustering. However, the theoretical properties of these methods remain largely unexplored. In the absence of statistical guarantees on the objective functions, it is difficult to determine if the algorithms optimizing the objectives will return good community structures. We investigate the consistency properties of the global optimizer of some of these objective functions under the multi-layer stochastic blockmodel. For this purpose, we derive several new asymptotic results showing consistency of the intermediate fusion techniques along with the spectral clustering of mean adjacency matrix under a high dimensional setup, where the number of nodes, the number of layers and the number of communities of the multi-layer graph grow. Our numerical study shows that the intermediate fusion techniques outperform late fusion methods, namely spectral clustering on aggregate spectral kernel and module allegiance matrix in sparse networks, while they outperform the spectral clustering of mean adjacency matrix in multi-layer networks that contain layers with both homophilic and heterophilic communities.

An Improved Forward Regression Variable Selection Algorithm for High-Dimensional Linear Regression Models

Variable selection plays an important role in various fields, such as process modeling and process monitoring. It generally involves a large number of predictor variables, usually with the number of predictor variables $d$ much larger than the sample size $n$ . Therefore, how to filter useful variables and extract useful information in high-dimensional setup is a critical issue in the era of big data. This paper proposes an improved Forward Regression algorithm for variable selection under the high-dimensional setup. The proposed improved Forward Regression method demonstrates good performance in relevant-variable selection by introducing a predefined stopping rule. The stopping rule links the residual sum of squares to the noise ratio so that the relevant predictors can be distinguished from the random noises. Throughout theoretical analysis and simulations, it is confirmed that the improved Forward Regression algorithm can identify relevant predictors to ensure selection consistency in variable selection. Compared with the traditional Forward Regression method, the proposed Forward Regression algorithm can improve prediction accuracy and reduce computational cost by selecting only the relevant variables.

New approach to Bayesian high-dimensional linear regression

Abstract Consider the problem of estimating parameters $X^n \in \mathbb{R}^n $, from $m$ response variables $Y^m = AX^n+Z^m$, under the assumption that the distribution of $X^n$ is known. Lack of computationally feasible algorithms that employ generic prior distributions and provide a good estimate of $X^n$ has limited the set of distributions researchers use to model the data. To address this challenge, in this article, a new estimation scheme named quantized maximum a posteriori (Q-MAP) is proposed. The new method has the following properties: (i) In the noiseless setting, it has similarities to maximum a posteriori (MAP) estimation. (ii) In the noiseless setting, when $X_1,\ldots,X_n$ are independent and identically distributed, asymptotically, as $n$ grows to infinity, its required sampling rate ($m/n$) for an almost zero-distortion recovery approaches the fundamental limits. (iii) It scales favorably with the dimensions of the problem and therefore is applicable to high-dimensional setups. (iv) The solution of the Q-MAP optimization can be found via a proposed iterative algorithm that is provably robust to error (noise) in response variables.

Sparse generalized principal component analysis for large-scale applications beyond Gaussianity

Principal Component Analysis (PCA) is a dimension reduction technique. It produces inconsistent estimators when the dimensionality is moderate to high, which is often the problem in modern large-scale applications where algorithm scalability and model interpretability are difficult to achieve, not to mention the prevalence of missing values. While existing sparse PCA methods alleviate inconsistency, they are constrained to the Gaussian assumption of classical PCA and fail to address algorithm scalability issues. We generalize sparse PCA to the broad exponential family distributions under high-dimensional setup, with built-in treatment for missing values. Meanwhile we propose a family of iterative sparse generalized PCA (SG-PCA) algorithms such that despite the non-convexity and non-smoothness of the optimization task, the loss function decreases in every iteration. In terms of ease and intuitive parameter tuning, our sparsity-inducing regularization is far superior to the popular Lasso. Furthermore, to promote overall scalability, accelerated gradient is integrated for fast convergence, while a progressive screening technique gradually squeezes out nuisance dimensions of a large-scale problem for feasible optimization. High-dimensional simulation and real data experiments demonstrate the efficiency and efficacy of SG-PCA.

Bernshteĭn–von Mises theorems for nonparametric function analysis via locally constant modelling: A unified approach

Various statistical models involve a certain function, say f, like the mean regression as a function of a covariate, the hazard rate as a function of time, the spectral density of a time series as a function of frequency, or an intensity as a function of geographical position, etc. Such functions are often modelled parametrically, whether for frequentist or Bayesian uses, and under weak conditions there are so-called Bernshteĭn–von Mises theorems implying that these two approaches are large-sample equivalent. Results of this nature do not necessarily hold up in nonparametric and high-dimensional setups, however. The aim of the present paper is to exhibit a unified framework and methodology for both frequentist and Bayesian nonparametric analysis, involving priors that set f constant over windows, and where the number m of such windows grows with sample size n. Applications include nonparametric regression, maximum likelihood with nonparametrically varying parameter functions, hazard rates being functions of covariates, and nonparametric analysis of stationary time series. We work out conditions on the number and sizes of the windows under which Bernshteĭn–von Mises type theorems can be established, with the prior changing with sample size via the growing number of windows. These conditions entail e.g. that if m∝nα, then α∈(14,12) is required. Illustrations of the general methodology are given, including setups with nonparametric regression, hazard rate estimation, and inference about frequency spectra for stationary time series.

Global Permutation Tests for Multivariate Ordinal Data: Alternatives, Test Statistics and The Null Dilemma

SummaryWe discuss two-sample global permutation tests for sets of multivariate ordinal data in possibly high dimensional set-ups, motivated by the analysis of data collected by means of the World Health Organization's ‘International classification of functioning, disability and health’. The tests do not require any modelling of the multivariate dependence structure. Specifically, we consider testing for marginal inhomogeneity and direction-independent marginal order. As opposed to max-T-tests, which are known to have good power against alternatives with few strong individual effects, the tests proposed have good power against alternatives with many weak individual effects. Permutation tests are valid only if the two multivariate distributions are identical under the null hypothesis. By means of simulations, we examine the practical effect of violations of this exchangeability condition. Our simulations suggest that theoretically invalid permutation tests can still be ‘practically valid’. In particular, they suggest that the degree of the permutation procedure's failure may be considered as a function of the difference in group-specific covariance matrices, the proportion between group sizes, the number of variables in the set, the test statistic used and the number of levels per variable.