Structure Of High-dimensional Data Research Articles

Inference for high-dimensional split-plot-designs: A unified approach for small to large numbers of factor levels

Statisticians increasingly face the problem to reconsider the adaptability of classical inference techniques. In particular, diverse types of high-dimensional data structures are observed in various research areas; disclosing the boundaries of conventional multivariate data analysis. Such situations occur, e.g., frequently in life sciences whenever it is easier or cheaper to repeatedly generate a large number $d$ of observations per subject than recruiting many, say $N$, subjects. In this paper, we discuss inference procedures for such situations in general heteroscedastic split-plot designs with $a$ independent groups of repeated measurements. These will, e.g., be able to answer questions about the occurrence of certain time, group and interactions effects or about particular profiles. The test procedures are based on standardized quadratic forms involving suitably symmetrized U-statistics-type estimators which are robust against an increasing number of dimensions $d$ and/or groups $a$. We then discuss their limit distributions in a general asymptotic framework and additionally propose improved small sample approximations. Finally, the small sample performance is investigated in simulations and applicability is illustrated by a real data analysis.

Integrative analysis reveals disrupted pathways regulated by microRNAs in cancer.

MicroRNAs (miRNAs) are small endogenous regulatory molecules that modulate gene expression post-transcriptionally. Although differential expression of miRNAs have been implicated in many diseases (including cancers), the underlying mechanisms of action remain unclear. Because each miRNA can target multiple genes, miRNAs may potentially have functional implications for the overall behavior of entire pathways. Here, we investigate the functional consequences of miRNA dysregulation through an integrative analysis of miRNA and mRNA expression data using a novel approach that incorporates pathway information a priori. By searching for miRNA-pathway associations that differ between healthy and tumor tissue, we identify specific relationships at the systems level which are disrupted in cancer. Our approach is motivated by the hypothesis that if an miRNA and pathway are associated, then the expression of the miRNA and the collective behavior of the genes in a pathway will be correlated. As such, we first obtain an expression-based summary of pathway activity using Isomap, a dimension reduction method which can articulate non-linear structure in high-dimensional data. We then search for miRNAs that exhibit differential correlations with the pathway summary between phenotypes as a means of finding aberrant miRNA-pathway coregulation in tumors. We apply our method to cancer data using gene and miRNA expression datasets from The Cancer Genome Atlas and compare ∼105 miRNA-pathway relationships between healthy and tumor samples from four tissues (breast, prostate, lung and liver). Many of the flagged pairs we identify have a biological basis for disruption in cancer.

Template matching and machine learning-based robust facial expression recognition system using multi-level Haar wavelet

ABSTRACTRecognition of facial expressions is important in industrial automation, security, medical, and many other fields. An image is a very rich and high dimensional data structure, which can result into a considerable computation when processed upon directly. Various feature extraction techniques have been proposed to represent the images efficiently in lower dimension which is understandable by the computer. In this paper, we propose Multi-Level Haar wavelet-based approach, which extracts salient features from prominent face regions at two different scales. The approach first segments most informative geometric components such as eye, mouth, etc. using the Adaboost cascade object detector. Segmented components are divided in M × N regions and feature vector is obtained by concatenating local Haar features extracted from each region. Feature vector is projected in Linear Discriminant Analysis space to reduce its size. For classification, we used template matching (Chi-Square and Cosine measure) and machine learning techniques (Logistic Regression and Support Vector Machine). Performance of proposed method is evaluated on various well-known data-sets like CK, Japanese Female Facial Expression, and Taiwanese Facial Expression Image Database. Adaptability of the feature is also tested on in-house Web-Enabled Spontaneous Facial Expression Data-set (WESFED). Comparison with state of the art method shows the superiority of proposed method.

Clustering of imbalanced high-dimensional media data

Media content in large repositories usually exhibits multiple groups of strongly varying sizes. Media of potential interest often form notably smaller groups. Such media groups differ so much from the remaining data that it may be worthy to look at them in more detail. In contrast, media with popular content appear in larger groups. Identifying groups of varying sizes is addressed by clustering of imbalanced data. Clustering highly imbalanced media groups is additionally challenged by the high dimensionality of the underlying features. In this paper, we present the imbalanced clustering (IClust) algorithm designed to reveal group structures in high-dimensional media data. IClust employs an existing clustering method in order to find an initial set of a large number of potentially highly pure clusters which are then successively merged. The main advantage of IClust is that the number of clusters does not have to be pre-specified and that no specific assumptions about the cluster or data characteristics need to be made. Experiments on real-world media data demonstrate that in comparison to existing methods, IClust is able to better identify media groups, especially groups of small sizes.

Testing Mutual Independence in High Dimension via Distance Covariance

SummaryWe introduce an L2-type test for testing mutual independence and banded dependence structure for high dimensional data. The test is constructed on the basis of the pairwise distance covariance and it accounts for the non-linear and non-monotone dependences among the data, which cannot be fully captured by the existing tests based on either Pearson correlation or rank correlation. Our test can be conveniently implemented in practice as the limiting null distribution of the test statistic is shown to be standard normal. It exhibits excellent finite sample performance in our simulation studies even when the sample size is small albeit the dimension is high and is shown to identify non-linear dependence in empirical data analysis successfully. On the theory side, asymptotic normality of our test statistic is shown under quite mild moment assumptions and with little restriction on the growth rate of the dimension as a function of sample size. As a demonstration of good power properties for our distance-covariance-based test, we further show that an infeasible version of our test statistic has the rate optimality in the class of Gaussian distributions with equal correlation.

A high-dimensional likelihood ratio test for circular symmetric covariance structure

ABSTRACTThe paper considers a high-dimensional hypothesis test on circular symmetric covariance structure. When both the dimension p and the sample size N tend to infinity with , it proves that under the assumption of Gaussian, the logarithmic likelihood ratio statistic converges in distribution to a Gaussian random variable, and the specific expressions of the mean and the variance are also obtained. The simulations indicate that our high-dimensional likelihood ratio method outperform those of traditional chi-square approximation method and high-dimensional edgeworth expansion method, and it is as effective as the more accurate high-dimensional edgeworth expansion method on analyzing the circular symmetric covariance structure of high-dimensional data.

Geochemical differentiation processes for arc magma of the Sengan volcanic cluster, Northeastern Japan, constrained from principal component analysis

In this study, with a view of understanding the structure of high-dimensional geochemical data and discussing the chemical processes at work in the evolution of arc magmas, we employed principal component analysis (PCA) to evaluate the compositional variations of volcanic rocks from the Sengan volcanic cluster of the Northeastern Japan Arc. We analyzed the trace element compositions of various arc volcanic rocks, sampled from 17 different volcanoes in a volcanic cluster. The PCA results demonstrated that the first three principal components accounted for 86% of the geochemical variation in the magma of the Sengan region. Based on the relationships between the principal components and the major elements, the mass–balance relationships with respect to the contributions of minerals, the composition of plagioclase phenocrysts, geothermal gradient, and seismic velocity structure in the crust, the first, the second, and the third principal components appear to represent magma mixing, crystallizations of olivine/pyroxene, and crystallizations of plagioclase, respectively. These represented 59%, 20%, and 6%, respectively, of the variance in the entire compositional range, indicating that magma mixing accounted for the largest variance in the geochemical variation of the arc magma. Our result indicated that crustal processes dominate the geochemical variation of magma in the Sengan volcanic cluster.

Ordered homogeneity pursuit lasso for group variable selection with applications to spectroscopic data

In high-dimensional data modeling, variable selection methods have been a popular choice to improve the prediction accuracy by effectively selecting the subset of informative variables, and such methods can enhance the model interpretability with sparse representation. In this study, we propose a novel group variable selection method named ordered homogeneity pursuit lasso (OHPL) that takes the homogeneity structure in high-dimensional data into account. OHPL is particularly useful in high-dimensional datasets with strongly correlated variables. We illustrate the approach using three real-world spectroscopic datasets and compare it with four state-of-the-art variable selection methods. The benchmark results on real-world data show that the proposed method is capable of identifying a small number of influential groups and has better prediction performance than its competitors. The OHPL method and the spectroscopic datasets are implemented and included in an R package OHPL available from https://ohpl.io.

Spherical Cap Packing Asymptotics and Rank-Extreme Detection

We study the spherical cap packing problem with a probabilistic approach. Such probabilistic considerations result in an asymptotic sharp universal uniform bound on the maximal inner product between any set of unit vectors and a stochastically independent uniformly distributed unit vector. When the set of unit vectors are themselves independently uniformly distributed, we further develop the extreme value distribution limit of the maximal inner product, which characterizes its uncertainty around the bound. As applications of the above asymptotic results, we derive (1) an asymptotic sharp universal uniform bound on the maximal spurious correlation, as well as its uniform convergence in distribution when the explanatory variables are independently Gaussian distributed; and (2) an asymptotic sharp universal bound on the maximum norm of a low-rank elliptically distributed vector, as well as related limiting distributions. With these results, we develop a fast detection method for a low-rank structure in high-dimensional Gaussian data without using the spectrum information.

CYCLOPS reveals human transcriptional rhythms in health and disease

Circadian rhythms modulate many aspects of physiology. Knowledge of the molecular basis of these rhythms has exploded in the last 20 years. However, most of these data are from model organisms, and translation to clinical practice has been limited. Here, we present an approach to identify molecular rhythms in humans from thousands of unordered expression measurements. Our algorithm, cyclic ordering by periodic structure (CYCLOPS), uses evolutionary conservation and machine learning to identify elliptical structure in high-dimensional data. From this structure, CYCLOPS estimates the phase of each sample. We validated CYCLOPS using temporally ordered mouse and human data and demonstrated its consistency on human data from two independent research sites. We used this approach to identify rhythmic transcripts in human liver and lung, including hundreds of drug targets and disease genes. Importantly, for many genes, the circadian variation in expression exceeded variation from genetic and other environmental factors. We also analyzed hepatocellular carcinoma samples and show these solid tumors maintain circadian function but with aberrant output. Finally, to show how this method can catalyze medical translation, we show that dosage time can temporally segregate efficacy from dose-limiting toxicity of streptozocin, a chemotherapeutic drug. In sum, these data show the power of CYCLOPS and temporal reconstruction in bridging basic circadian research and clinical medicine.

Restricted Isometry Property of Gaussian Random Projection for Finite Set of Subspaces

Dimension reduction plays an essential role when decreasing the complexity of solving large-scale problems. The well-known Johnson-Lindenstrauss (JL) Lemma and Restricted Isometry Property (RIP) admit the use of random projection to reduce the dimension while keeping the Euclidean distance, which leads to the boom of Compressed Sensing and the field of sparsity related signal processing. Recently, successful applications of sparse models in computer vision and machine learning have increasingly hinted that the underlying structure of high dimensional data looks more like a union of subspaces (UoS). In this paper, motivated by JL Lemma and an emerging field of Compressed Subspace Clustering (CSC), we study for the first time the RIP of Gaussian random matrices for the compression of two subspaces based on the generalized projection $F$-norm distance. We theoretically prove that with high probability the affinity or distance between two projected subspaces are concentrated around their estimates. When the ambient dimension after projection is sufficiently large, the affinity and distance between two subspaces almost remain unchanged after random projection. Numerical experiments verify the theoretical work.

Subspace clustering using a symmetric low-rank representation

In this paper, we propose a low-rank representation with symmetric constraint (LRRSC) method for robust subspace clustering. Given a collection of data points approximately drawn from multiple subspaces, the proposed technique can simultaneously recover the dimension and members of each subspace. LRRSC extends the original low-rank representation algorithm by integrating a symmetric constraint into the low-rankness property of high-dimensional data representation. The symmetric low-rank representation, which preserves the subspace structures of high-dimensional data, guarantees weight consistency for each pair of data points so that highly correlated data points of subspaces are represented together. Moreover, it can be efficiently calculated by solving a convex optimization problem. We provide a proof for minimizing the nuclear-norm regularized least square problem with a symmetric constraint. The affinity matrix for spectral clustering can be obtained by further exploiting the angular information of the principal directions of the symmetric low-rank representation. This is a critical step towards evaluating the memberships between data points. Besides, we also develop eLRRSC algorithm to improve the scalability of the original LRRSC by considering its closed form solution. Experimental results on benchmark databases demonstrate the effectiveness and robustness of LRRSC and its variant compared with several state-of-the-art subspace clustering algorithms.

HB-File: An efficient and effective high-dimensional big data storage structure based on US-ELM

With the rapid development of computer and the Internet techniques, the amount of data in all walks of life increases sharply, especially accumulating numerous high-dimensional big data such as the network transactions data, the user reviews data and the multimedia data. High-dimensional big data mixes the typical features of both high-dimensional data and big data, which has also brought new problems and great challenges for processing and optimizing the high-dimensional big data. In this case, the storage structure of high-dimensional big data is a critical factor that can affect the processing performance in a fundamental way. However, due to the huge dimensionality feature of high-dimensional data, the existing data storage techniques, such as row-store and column-store, are not very suitable for high-dimensional and large scale data. Therefore, in this paper, we present an efficient high-dimensional big data storage structure based on US-ELM, High-dimensional Big Data File, named HB-File. Then, we propose a fuzzy cluster algorithm to differentiate the key dimension and non-key dimension of high-dimensional big data based on US-ELM, which can also gain the clusters of key dimension. After that, we propose the execution and API of HB-File based on the open source implementation of MapReduce, Hadoop system. With the intensive experiments, we show the effectiveness of HB-File in satisfying the storage of high-dimensional big data.

Missing value imputation for gene expression data by tailored nearest neighbors.

High dimensional data like gene expression and RNA-sequences often contain missing values. The subsequent analysis and results based on these incomplete data can suffer strongly from the presence of these missing values. Several approaches to imputation of missing values in gene expression data have been developed but the task is difficult due to the high dimensionality (number of genes) of the data. Here an imputation procedure is proposed that uses weighted nearest neighbors. Instead of using nearest neighbors defined by a distance that includes all genes the distance is computed for genes that are apt to contribute to the accuracy of imputed values. The method aims at avoiding the curse of dimensionality, which typically occurs if local methods as nearest neighbors are applied in high dimensional settings. The proposed weighted nearest neighbors algorithm is compared to existing missing value imputation techniques like mean imputation, KNNimpute and the recently proposed imputation by random forests. We use RNA-sequence and microarray data from studies on human cancer to compare the performance of the methods. The results from simulations as well as real studies show that the weighted distance procedure can successfully handle missing values for high dimensional data structures where the number of predictors is larger than the number of samples. The method typically outperforms the considered competitors.

An Enhanced Visualization Method to Aid Behavioral Trajectory Pattern Recognition Infrastructure for Big Longitudinal Data.

Big longitudinal data provide more reliable information for decision making and are common in all kinds of fields. Trajectory pattern recognition is in an urgent need to discover important structures for such data. Developing better and more computationally-efficient visualization tool is crucial to guide this technique. This paper proposes an enhanced projection pursuit (EPP) method to better project and visualize the structures (e.g. clusters) of big high-dimensional (HD) longitudinal data on a lower-dimensional plane. Unlike classic PP methods potentially useful for longitudinal data, EPP is built upon nonlinear mapping algorithms to compute its stress (error) function by balancing the paired weights for between and within structure stress while preserving original structure membership in the high-dimensional space. Specifically, EPP solves an NP hard optimization problem by integrating gradual optimization and non-linear mapping algorithms, and automates the searching of an optimal number of iterations to display a stable structure for varying sample sizes and dimensions. Using publicized UCI and real longitudinal clinical trial datasets as well as simulation, EPP demonstrates its better performance in visualizing big HD longitudinal data.

Testing block-diagonal covariance structure for high-dimensional data under non-normality

In this article, we propose a test for making an inference about the block-diagonal covariance structure of a covariance matrix in non-normal high-dimensional data. We prove that the limiting null distribution of the proposed test is normal under mild conditions when its dimension is substantially larger than its sample size. We further study the local power of the proposed test. Finally, we study the finite-sample performance of the proposed test via Monte Carlo simulations. We demonstrate the relevance and benefits of the proposed approach for a number of alternative covariance structures.

A locality constrained self-representation approach for unsupervised feature selection

Recently, regularised self-representation (RSR) has been proposed as an efficient unsupervised feature selection algorithm. However, RSR only takes the self-representation ability of features into account, and neglects the locality structure preserving ability of features, which may degrade its performance. To overcome this limitation, a novel algorithm termed locality constrained regularised self-representation (LCRSR) is proposed in this paper. In our algorithm, a local scatter matrix is introduced to encode the locality geometric structure of high-dimensional data. Therefore, the locality information of the input database can be well preserved. Moreover, a simple yet efficient iterative update algorithm is developed to solve the proposed LCRSR. Extensive experiments are conducted on five publicly available databases (such as JAFFE, ORL, AR, COIL20 and SRBCT) to demonstrate the efficiency of the proposed algorithm. Experimental results show that LCRSR obtains better clustering performance than some other state-of-the-art approaches.

Classification and dimensional reduction using restricted radial basis function networks

Visualizing high dimensional data by projecting them into two or three dimensional space is one of the most effective ways to intuitively understand the data's underlying characteristics, for example their class neighborhood structure. While data visualization in low dimensional space can be efficient for revealing the data's underlying characteristics, classifying a new sample in the reduced-dimensional space is not always beneficial because of the loss of information in expressing the data. It is possible to classify the data in the high dimensional space, while visualizing them in the low dimensional space, but in this case, the visualization is often meaningless because it fails to illustrate the underlying characteristics that are crucial for the classification process. In this paper, the performance-preserving property of the previously proposed Restricted Radial Basis Function Network in reducing the dimension of labeled data is explained. Here, it is argued through empirical experiments that the internal representation of the Restricted Radial Basis Function Network, which during the supervised learning process organizes a visualizable two dimensional map, does not only preserve the topographical structure of high dimensional data but also captures their class neighborhood structures that are important for classifying them. Hence, unlike many of the existing dimension reduction methods, the Restricted Radial Basis Function Network offers two dimensional visualization that is strongly correlated with the classification process.

Analysis of human motion variation patterns using UMPCA

The rapid development of motion capture technologies has greatly increased the use of human motion data in many applications. This has increased the demand to have an effective means to systematically analyze those massive data in order to understand human motion variation patterns. This paper studies one typical type of motion data, which are recorded as multi-stream trajectories of human joints. Such a high dimensional multi-stream data structure makes it difficult to directly perform visual comparisons or simply apply conventional methods such as PCA to capture the variation of human motion patterns. In this paper, a high order array (tensor) is suggested for data representation, based on which the Uncorrelated Multilinear Principal Component Analysis (UMPCA) is applied to analyze the variation of human motion patterns. A simulation study is presented to show the superiority of UMPCA over PCA in preserving the cross-correlation among multi-stream trajectories. The effectiveness of UMPCA is also demonstrated using a case study for analyzing vehicle ingress test data.

Context Specific and Differential Gene Co-expression Networks via Bayesian Biclustering.

Identifying latent structure in high-dimensional genomic data is essential for exploring biological processes. Here, we consider recovering gene co-expression networks from gene expression data, where each network encodes relationships between genes that are co-regulated by shared biological mechanisms. To do this, we develop a Bayesian statistical model for biclustering to infer subsets of co-regulated genes that covary in all of the samples or in only a subset of the samples. Our biclustering method, BicMix, allows overcomplete representations of the data, computational tractability, and joint modeling of unknown confounders and biological signals. Compared with related biclustering methods, BicMix recovers latent structure with higher precision across diverse simulation scenarios as compared to state-of-the-art biclustering methods. Further, we develop a principled method to recover context specific gene co-expression networks from the estimated sparse biclustering matrices. We apply BicMix to breast cancer gene expression data and to gene expression data from a cardiovascular study cohort, and we recover gene co-expression networks that are differential across ER+ and ER- samples and across male and female samples. We apply BicMix to the Genotype-Tissue Expression (GTEx) pilot data, and we find tissue specific gene networks. We validate these findings by using our tissue specific networks to identify trans-eQTLs specific to one of four primary tissues.