High-dimensional Multivariate Data Research Articles

MTS2Graph: Interpretable multivariate time series classification with temporal evolving graphs

Conventional time series classification approaches based on bags of patterns or shapelets face significant challenges in dealing with a vast amount of feature candidates from high-dimensional multivariate data. In contrast, deep neural networks can learn low-dimensional features efficiently, and in particular, convolutional neural networks have shown promising results in classifying multivariate time series data. A key factor in the success of deep neural networks is this astonishing expressive power. However, this power comes at the cost of complex, black-boxed models, conflicting with the goals of building reliable and human-understandable models. In this work11The code can be found at https://github.com/raneeny/MTS2Graph., we introduce a new interpretable framework for multivariate time series data that by extracting and clustering the input quantifies the contribution of time-varying input variables and each signal’s role to the classification. We construct a graph that captures the temporal relationship between the extracted patterns for each layer and propose an effective merging strategy to aggregate those graphs into one. Finally, a graph embedding algorithm generates new representations of the created interpretable time-series features. Our extensive experiments indicate the benefit of our time-aware graph-based representation in multivariate time series classification while enriching them with more interpretability.

Cglasso: An R Package for Conditional Graphical Lasso Inference with Censored and Missing Values

Sparse graphical models have revolutionized multivariate inference. With the advent of high-dimensional multivariate data in many applied fields, these methods are able to detect a much lower-dimensional structure, often represented via a sparse conditional independence graph. There have been numerous extensions of such methods in the past decade. Many practical applications have additional covariates or suffer from missing or censored data. Despite the development of these extensions of sparse inference methods for graphical models, there have been so far no implementations for, e.g., conditional graphical models. Here we present the general-purpose package cglasso for estimating sparse conditional Gaussian graphical models with potentially missing or censored data. The method employs an efficient expectation-maximization estimation of an ℓ₁ -penalized likelihood via a block-coordinate descent algorithm. The package has a user-friendly data manipulation interface. It estimates a solution path and includes various automatic selection algorithms for the two ℓ₁ tuning parameters, associated with the sparse precision matrix and sparse regression coefficients, respectively. The package pays particular attention to the visualization of the results, both by means of marginal tables and figures, and of the inferred conditional independence graphs. This package provides a unique and computational efficient implementation of a conditional Gaussian graphical model that is able to deal with the additional complications of missing and censored data. As such it constitutes an important contribution for empirical scientists wishing to detect sparse structures in high-dimensional data.

Forecasting daily emergency department arrivals using high-dimensional multivariate data: a feature selection approach

Background and objectiveEmergency Department (ED) overcrowding is a chronic international issue that is associated with adverse treatment outcomes. Accurate forecasts of future service demand would enable intelligent resource allocation that could alleviate the problem. There has been continued academic interest in ED forecasting but the number of used explanatory variables has been low, limited mainly to calendar and weather variables. In this study we investigate whether predictive accuracy of next day arrivals could be enhanced using high number of potentially relevant explanatory variables and document two feature selection processes that aim to identify which subset of variables is associated with number of next day arrivals. Performance of such predictions over longer horizons is also shown.MethodsWe extracted numbers of total daily arrivals from Tampere University Hospital ED between the time period of June 1, 2015 and June 19, 2019. 158 potential explanatory variables were collected from multiple data sources consisting not only of weather and calendar variables but also an extensive list of local public events, numbers of website visits to two hospital domains, numbers of available hospital beds in 33 local hospitals or health centres and Google trends searches for the ED. We used two feature selection processes: Simulated Annealing (SA) and Floating Search (FS) with Recursive Least Squares (RLS) and Least Mean Squares (LMS). Performance of these approaches was compared against autoregressive integrated moving average (ARIMA), regression with ARIMA errors (ARIMAX) and Random Forest (RF). Mean Absolute Percentage Error (MAPE) was used as the main error metric.ResultsCalendar variables, load of secondary care facilities and local public events were dominant in the identified predictive features. RLS-SA and RLS-FA provided slightly better accuracy compared ARIMA. ARIMAX was the most accurate model but the difference between RLS-SA and RLS-FA was not statistically significant.ConclusionsOur study provides new insight into potential underlying factors associated with number of next day presentations. It also suggests that predictive accuracy of next day arrivals can be increased using high-dimensional feature selection approach when compared to both univariate and nonfiltered high-dimensional approach. Performance over multiple horizons was similar with a gradual decline for longer horizons. However, outperforming ARIMAX remains a challenge when working with daily data. Future work should focus on enhancing the feature selection mechanism, investigating its applicability to other domains and in identifying other potentially relevant explanatory variables.

Multivariate functional group sparse regression: Functional predictor selection.

In this paper, we propose methods for functional predictor selection and the estimation of smooth functional coefficients simultaneously in a scalar-on-function regression problem under a high-dimensional multivariate functional data setting. In particular, we develop two methods for functional group-sparse regression under a generic Hilbert space of infinite dimension. We show the convergence of algorithms and the consistency of the estimation and the selection (oracle property) under infinite-dimensional Hilbert spaces. Simulation studies show the effectiveness of the methods in both the selection and the estimation of functional coefficients. The applications to functional magnetic resonance imaging (fMRI) reveal the regions of the human brain related to ADHD and IQ.

Low-Rank Characteristic Tensor Density Estimation Part I: Foundations

Effective non-parametric density estimation is a key challenge in high-dimensional multivariate data analysis. In this paper,we propose a novel approach that builds upon tensor factorization tools. Any multivariate density can be represented by its characteristic function, via the Fourier transform. If the sought density is compactly supported, then its characteristic function can be approximated, within controllable error, by a finite tensor of leading Fourier coefficients, whose size de-pends on the smoothness of the underlying density. This tensor can be naturally estimated from observed realizations of the random vector of interest, via sample averaging. In order to circumvent the curse of dimensionality, we introduce a low-rank model of this characteristic tensor, which significantly improves the density estimate especially for high-dimensional data and/or in the sample-starved regime. By virtue of uniqueness of low-rank tensor decomposition, under certain conditions, our method enables learning the true data-generating distribution. We demonstrate the very promising performance of the proposed method using several measured datasets.

A high dimensional nonparametric test for proportional covariance matrices

This work is concerned with testing the proportionality between two high dimensional covariance matrices. Several tests for proportional covariance matrices, based on modifying the classical likelihood ratio test and applicable in high dimension, have been proposed in the literature. Despite their usefulness, they tend to have unsatisfactory performance for nonnormal high dimensional multivariate data in terms of size or power. This article proposes a new high dimensional test by developing a bias correction to the existing test statistic constructed based on a scaled distance measure. The suggested test is nonparametric without requiring any specific parametric distribution such as the normality assumption. It can accommodate scenarios where the data dimension p is greater than the sample size n, namely the “large p, small n” problem. With the aid of tools in modern probability theory, we study theoretical properties of the newly proposed test, which include the asymptotic normality and a power evaluation. We demonstrate empirically that our proposal has good size and power performances for a range of dimensions, sample sizes and distributions in comparison with the existing counterparts.

Spatial factor modeling: A Bayesian matrix-normal approach for misaligned data.

Multivariate spatially oriented data sets are prevalent in the environmental and physical sciences. Scientists seek to jointly model multiple variables, each indexed by a spatial location, to capture any underlying spatial association for each variable and associations among the different dependent variables. Multivariate latent spatial process models have proved effective in driving statistical inference and rendering better predictive inference at arbitrary locations for the spatial process. High-dimensional multivariate spatial data, which are the theme of this article, refer to data sets where the number of spatial locations and the number of spatially dependent variables is very large. The field has witnessed substantial developments in scalable models for univariate spatial processes, but such methods for multivariate spatial processes, especially when the number of outcomes are moderately large, are limited in comparison. Here, we extend scalable modeling strategies for a single process to multivariate processes. We pursue Bayesian inference, which is attractive for full uncertainty quantification of the latent spatial process. Our approach exploits distribution theory for the matrix-normal distribution, which we use to construct scalable versions of a hierarchical linear model of coregionalization (LMC) and spatial factor models that deliver inference over a high-dimensional parameter space including the latent spatial process. We illustrate the computational and inferential benefits of our algorithms over competing methods using simulation studies and an analysis of a massive vegetation index dataset.

Two-sample test in high dimensions through random selection

Testing the equality for two-sample means with high dimensional distributions is a fundamental problem in statistics. In the past two decades, many efforts have been devoted to comparing the mean vectors of two populations. Many existing tests rely on naive diagonal or trace estimators of the covariance matrix, ignoring the dependence structure between variables. To make more use of the dependence structure, a new nonparametric test based on random selections is proposed to test the population mean vector of nonnormal high-dimensional multivariate data. This makes more efficient use of the covariance structure to deal with dependent variables. The asymptotic null distribution of the proposed test is standard normal, regardless of the parent distributions of the random samples and the relations between data dimensions and sample sizes. Extensive simulations show that the power performance of the proposed test is encouraging compared with some existing methods.

TapNet: Multivariate Time Series Classification with Attentional Prototypical Network

With the advance of sensor technologies, the Multivariate Time Series classification (MTSC) problem, perhaps one of the most essential problems in the time series data mining domain, has continuously received a significant amount of attention in recent decades. Traditional time series classification approaches based on Bag-of-Patterns or Time Series Shapelet have difficulty dealing with the huge amounts of feature candidates generated in high-dimensional multivariate data but have promising performance even when the training set is small. In contrast, deep learning based methods can learn low-dimensional features efficiently but suffer from a shortage of labelled data. In this paper, we propose a novel MTSC model with an attentional prototype network to take the strengths of both traditional and deep learning based approaches. Specifically, we design a random group permutation method combined with multi-layer convolutional networks to learn the low-dimensional features from multivariate time series data. To handle the issue of limited training labels, we propose a novel attentional prototype network to train the feature representation based on their distance to class prototypes with inadequate data labels. In addition, we extend our model into its semi-supervised setting by utilizing the unlabeled data. Extensive experiments on 18 datasets in a public UEA Multivariate time series archive with eight state-of-the-art baseline methods exhibit the effectiveness of the proposed model.

Testing independence in high-dimensional multivariate normal data

A simple statistic was proposed for testing the complete independence of components of random vector having a multivariate normal distribution. Using the martingale central limit theorem, it was proved that this test statistic converges in distribution to the normal as long as both the sample size and dimension go to infinity. Simulations were also carried out to performance evaluation of the proposed test for dealing with the high-dimensional data and the results were compared to those of the state-of-the-art tests. The results confirmed that, in terms of the average relative error criterion, the proposed test has adequate type I error rate control and comparable power to those tests with smaller size distortions.

High-Dimensional Multivariate Time Series Data Visualization Using an Open Source Framework

Multivariate Time Series plays a major part in statistics, signal processing, pattern recognition, econometrics, weather forecasting and earthquake prediction etc. It is very tedious task to select the appropriate technique to visualize the high-dimensional data in order to get insight or meaningful information. In this paper, we used an Open Source package named as Visbrain to visualize high-dimensional multivariate healthcare data in optimized way. It involves two dimensions of reflection: (1) questions that speak to exceedingly configurable visual natives (availability of EEG and ECG sources, and so forth.) and (2) graphical UIs for more elevated amount connections. The article level offers adaptable and measured devices for delivering and computerizing the generation of numbers with a comparative way to deal with that of Matplotlib with subplots. The second dimension outwardly associates these articles by controlling properties and communications by means of graphical interfaces. The present form of Visbrain (rendition 0.4.4) contains 14 distinct items and three custom graphical interfaces, worked with PyQt: Signal, for the control of fleeting and ghostly properties. Every module has been created in close cooperation with end clients, mostly neuroscientists and area specialists, who utilize their experience to make Visbrain as straightforward as feasible for the chronicle modalities (eg intracranial EEG, ECG, scalp EEG, MEG, anatomical and useful MRI). The paper disscuses the features and various modules for current understanding research trends in the field of Visual Analytics.

Bayesian analysis of longitudinal quality of life measures with informative missing data using a selection model.

Health-related quality of life consists of multi-dimensional measurements of physical and mental health domains. Health-related quality of life is often followed up to evaluate efficacy of treatments in clinical studies. During the follow-up period, a missing data problem inevitably arises. When missing data occur for reasons related to poor health-related quality of life, a complete-case only analysis can lead to invalid inferences. We propose a Bayesian approach to analyze longitudinal moderate to high-dimensional multivariate outcome data in the presence of non-ignorable missing data. To account for non-ignorable missing data, we employ a selection model for the joint likelihood factorization where we apply Bayesian spike and slab variable selection in the missing data mechanism to detect informative factors among multiple outcomes. We model the relationship between multiple outcomes and covariates using linear mixed effects models where multiple outcome correlations are captured by a hierarchical structure. We conduct simulation studies to evaluate the performance of the proposed method compared with the conventional last observation carried forward approach. We use a motivating example that originates from a longitudinal study of quality of life in gastric cancer patients who underwent distal gastrectomy. In this application, we demonstrate that our proposed method can offer efficiency gain in the marginal associations and provide the associations between outcomes and the absence of patients' information.

The Graphical Horseshoe Estimator for Inverse Covariance Matrices

We develop a new estimator of the inverse covariance matrix for high-dimensional multivariate normal data using the horseshoe prior. The proposed graphical horseshoe estimator has attractive properties compared to other popular estimators, such as the graphical lasso and the graphical smoothly clipped absolute deviation. The most prominent benefit is that when the true inverse covariance matrix is sparse, the graphical horseshoe provides estimates with small information divergence from the sampling model. The posterior mean under the graphical horseshoe prior can also be almost unbiased under certain conditions. In addition to these theoretical results, we also provide a full Gibbs sampler for implementing our estimator. MATLAB code is available for download from github at http://github.com/liyf1988/GHS. The graphical horseshoe estimator compares favorably to existing techniques in simulations and in a human gene network data analysis. Supplementary materials for this article are available online.

Distributed Based Serial Regression Multiple Imputation for High Dimensional Multivariate Data in Multicore Environment of Cloud

Multiple imputations (MI) are predominantly applied in such processes that are involved in the transaction of huge chunks of missing data. Multivariate data that follow traditional statistical models undergoes great suffering for the inadequate availability of pertinent data. The field of distributed computing research faces the biggest hurdle in the form of insufficient high dimensional multivariate data. It mainly deals with the analysis of parallel input problems found in the cloud computing network in general and evaluation of high-performance computing in particular. In fact, it is a tough task to utilize parallel multiple input methods for accomplishing remarkable performance as well as allowing huge datasets achieves scale. In this regard, it is essential that a credible data system is developed and a decomposition strategy is used to partition workload in the entire process for minimum data dependence. Subsequently, a moderate synchronization and/or meager communication liability is followed for placing parallel impute methods for achieving scale as well as more processes. The present article proposes many novel applications for better efficiency. As the first step, this article suggests distributed-oriented serial regression multiple imputation for enhancing the efficiency of imputation task in high dimensional multivariate normal data. As the next step, the processes done in three diverse with parallel back ends viz. Multiple imputation that used the socket method to serve serial regression and the Fork Method to distribute work over workers, and also same work experiments in dynamic structure with a load balance mechanism. In the end, the set of distributed MI methods are used to experimentally analyze amplitude of imputation scores spanning across three probable scenarios in the range of 1:500. Further, the study makes an important observation that due to the efficiency of numerous imputation methods, the data is arranged proportionately in a missing range of 10% to 50%, low to high, while dealing with data between 1000 and 100,000 samples. The experiments are done in a cloud environment and demonstrate that it is possible to generate a decent speed by lessening the repetitive communication between processors.

Bayesian variable selection for multivariate zero-inflated models: Application to microbiome count data.

Microorganisms play critical roles in human health and disease. They live in diverse communities in which they interact synergistically or antagonistically. Thus for estimating microbial associations with clinical covariates, such as treatment effects, joint (multivariate) statistical models are preferred. Multivariate models allow one to estimate and exploit complex interdependencies among multiple taxa, yielding more powerful tests of exposure or treatment effects than application of taxon-specific univariate analyses. Analysis of microbial count data also requires special attention because data commonly exhibit zero inflation, i.e., more zeros than expected from a standard count distribution. To meet these needs, we developed a Bayesian variable selection model for multivariate count data with excess zeros that incorporates information on the covariance structure of the outcomes (counts for multiple taxa), while estimating associations with the mean levels of these outcomes. Though there has been much work on zero-inflated models for longitudinal data, little attention has been given to high-dimensional multivariate zero-inflated data modeled via a general correlation structure. Through simulation, we compared performance of the proposed method to that of existing univariate approaches, for both the binary ("excess zero") and count parts of the model. When outcomes were correlated the proposed variable selection method maintained type I error while boosting the ability to identify true associations in the binary component of the model. For the count part of the model, in some scenarios the univariate method had higher power than the multivariate approach. This higher power was at a cost of a highly inflated false discovery rate not observed with the proposed multivariate method. We applied the approach to oral microbiome data from the Pediatric HIV/AIDS Cohort Oral Health Study and identified five (of 44) species associated with HIV infection.

Sparse-Group Independent Component Analysis with application to yield curves prediction

We propose a Sparse-Group Independent Component Analysis (SG-ICA) method to extract independent factors from high dimensional multivariate data. The method provides a unified and flexible framework that automatically identifies the number of factors and simultaneously estimates a sparse loading matrix, enables us to discover important features and offers improved interpretability of the estimators. We establish the consistency and asymptotic normality of the loading matrix estimator, demonstrate its finite sample performance with simulation studies, and illustrate its application using the daily US Overnight Index Swap rates from Oct 2011 to Mar 2015 with 15 maturities ranging from 1 week to 30 years. With higher efficiency of extracting factors, the forecasting performance of the SG-ICA is remarkably better than the popular parametric DNS model in an era of quantitative easing with short-term interest rate being close to zero.

A Penalized Likelihood Framework for High-Dimensional Phylogenetic Comparative Methods and an Application to New-World Monkeys Brain Evolution.

Working with high-dimensional phylogenetic comparative data sets is challenging because likelihood-based multivariate methods suffer from low statistical performances as the number of traits $p $ approaches the number of species $n $ and because some computational complications occur when $p $ exceeds $n$. Alternative phylogenetic comparative methods have recently been proposed to deal with the large $p $ small $n $ scenario but their use and performances are limited. Herein, we develop a penalized likelihood (PL) framework to deal with high-dimensional comparative data sets. We propose various penalizations and methods for selecting the intensity of the penalties. We apply this general framework to the estimation of parameters (the evolutionary trait covariance matrix and parameters of the evolutionary model) and model comparison for the high-dimensional multivariate Brownian motion (BM), Early-burst (EB), Ornstein-Uhlenbeck (OU), and Pagel's lambda models. We show using simulations that our PL approach dramatically improves the estimation of evolutionary trait covariance matrices and model parameters when $p$ approaches $n$, and allows for their accurate estimation when $p$ equals or exceeds $n$. In addition, we show that PL models can be efficiently compared using generalized information criterion (GIC). We implement these methods, as well as the related estimation of ancestral states and the computation of phylogenetic principal component analysis in the R package RPANDA and mvMORPH. Finally, we illustrate the utility of the new proposed framework by evaluating evolutionary models fit, analyzing integration patterns, and reconstructing evolutionary trajectories for a high-dimensional 3D data set of brain shape in the New World monkeys. We find a clear support for an EB model suggesting an early diversification of brain morphology during the ecological radiation of the clade. PL offers an efficient way to deal with high-dimensional multivariate comparative data.

Pattern Clustering of Hysteresis Time Series With Multivalued Mapping Using Tensor Decomposition

Many actuators and sensors made by smart materials have hysteresis feature which is a complex nonlinear phenomenon with multivalued mapping. Accurate identification of hysteresis pattern in those sensors and actuators is helpful for improving the modeling and control strategies of the systems. In this paper, a general framework of the pattern clustering of hysteretic time series is developed on the basis of tensor decomposition. First, high-dimensional multivariate data of hysteresis objects are transformed into three-order hysteresis tensors. Then the multilinear principal component analysis (MPCA) is utilized to reduce the dimensionality of hysteresis tensors. Afterward, a novel tensor ${k}$ -means clustering (CTKmeans) based on tensor distance and cycle variation feature initialization is developed for the clustering of tensor objects. In order to evaluate the performance of the proposed approach, the experiment of hysteresis feature test using polyvinylidene fluoride (PVDF)-based pressure sensors is implemented. The measured high-dimensional hysteresis time series of PVDF successively undergoes the procedures of filtering, segmentation, MPCA dimensionality reduction and CTKmeans clustering. The experimental results show that the MPCA can capture more significant inherent features of hysteresis objects than the principal component analysis (PCA) or the kernel PCA in the dimensionality reduction. In terms of tensor distance and cycle variation feature initialization, the CTKmeans outperforms the random-initialization-based tensor ${k}$ -means and standard ${k}$ -means in the clustering of hysteresis tensors. With the tensor decomposition approach of multivariate time series, the hysteresis objects with nonlinear multivalued mapping features can be effectively identified by the combination of MPCA and CTKmeans.

Interactive and coordinated visualization approaches for biological data analysis.

The field of computational biology has become largely dependent on data visualization tools to analyze the increasing quantities of data gathered through the use of new and growing technologies. Aside from the volume, which often results in large amounts of noise and complex relationships with no clear structure, the visualization of biological data sets is hindered by their heterogeneity, as data are obtained from different sources and contain a wide variety of attributes, including spatial and temporal information. This requires visualization approaches that are able to not only represent various data structures simultaneously but also provide exploratory methods that allow the identification of meaningful relationships that would not be perceptible through data analysis algorithms alone. In this article, we present a survey of visualization approaches applied to the analysis of biological data. We focus on graph-based visualizations and tools that use coordinated multiple views to represent high-dimensional multivariate data, in particular time series gene expression, protein-protein interaction networks and biological pathways. We then discuss how these methods can be used to help solve the current challenges surrounding the visualization of complex biological data sets.

Selective factor extraction in high dimensions

SUMMARY This paper studies simultaneous feature selection and extraction in supervised and unsupervised learning. We propose and investigate selective reduced rank regression for constructing optimal explanatory factors from a parsimonious subset of input features. The proposed estimators enjoy sharp oracle inequalities, and with a predictive information criterion for model selection, they adapt to unknown sparsity by controlling both rank and row support of the coefficient matrix. A class of algorithms is developed that can accommodate various convex and nonconvex sparsity-inducing penalties, and can be used for rank-constrained variable screening in high-dimensional multivariate data. The paper also showcases applications in macroeconomics and computer vision to demonstrate how low-dimensional data structures can be effectively captured by joint variable selection and projection.