Multivariate Data Structures Research Articles

Multivariate pattern analysis: a method and software to reveal, quantify, and visualize predictive association patterns in multicollinear data

BackgroundStrongly multicollinear covariates, such as those typically represented in metabolomics applications, represent a challenge for multivariate regression analysis. These challenges are commonly circumvented by reducing the number of covariates to a subset of linearly independent variables, but this strategy may lead to loss of resolution and thus produce models with poorer interpretative potential. The aim of this work was to implement and illustrate a method, multivariate pattern analysis (MVPA), which can handle multivariate covariates without compromising resolution or model quality.ResultsMVPA has been implemented in an open-source R package of the same name, mvpa. To facilitate the usage and interpretation of complex association patterns, mvpa has also been integrated into an R shiny app, mvpaShiny, which can be accessed on www.mvpashiny.org. MVPA utilizes a general projection algorithm that embraces a diversity of possible models. The method handles multicollinear and even linear dependent covariates. MVPA separates the variance in the data into orthogonal parts within the frame of a single joint model: one part describing the relations between covariates, outcome, and explanatory variables and another part describing the “net” predictive association pattern between outcome and explanatory variables. These patterns are visualized and interpreted in variance plots and plots for pattern analysis and ranking according to variable importance. Adjustment for a linear dependent covariate is performed in three steps. First, partial least squares regression with repeated Monte Carlo resampling is used to determine the number of predictive PLS components for a model relating the covariate to the outcome. Second, postprocessing of this PLS model by target projection provided a single component expressing the predictive association pattern between the outcome and the covariate. Third, the outcome and the explanatory variables were adjusted for the covariate by using the target score in the projection algorithm to obtain “net” data. We illustrate the main features of MVPA by investigating the partial mediation of a linearly dependent metabolomics descriptor on the association pattern between a measure of insulin resistance and lifestyle-related factors.ConclusionsOur method and implementation in R extend the range of possible analyses and visualizations that can be performed for complex multivariate data structures. The R packages are available on github.com/liningtonlab/mvpa and github.com/liningtonlab/mvpaShiny.

Uluslararası Öğrenci Değerlendirme Programı skorlarına göre ülkelerin kümelenmesi

This study aims to cluster 65 countries based on PISA results. In the study, PISA results (Science-Mathematics-Reading) published by OECD in 2015 and 2018 were used. The main purpose of the analysis is to apply cluster analysis using a multivariate data structure to identify similarities and differences in education systems between countries. In this analysis, the k-means method and the hierarchical clustering algorithm were used to group countries into specific groups, so that countries with similar educational performance were included in the same cluster. In addition, Dunn, Connectivity and Silhouette indexes were used to increase the reliability of the analysis and to determine the optimal number of clusters. According to the validation indexes, k-means method with k=2 was used for 2015 PISA scores while hierarchical clustering algorithm with k=2 was used for 2018 PISA scores. In 2015, Turkey was the only country that changed clusters between the countries clustered according to their PISA scores and the countries clustered according to their PISA scores in 2018, and the reasons for this change were discussed. It is also observed that Turkey was in Cluster-1 in 2015, which includes countries with lower performance, and in Cluster-2 in 2018, which includes countries with higher performance. The clustering methods and indexes used provide a more robust and informed interpretation of the results obtained and make an important contribution to understanding the education systems of countries based on PISA results and grouping countries with similar performance.

Bayesian Learning of Graph Substructures

Graphical models provide a powerful methodology for learning the conditional independence structure in multivariate data. Inference is often focused on estimating individual edges in the latent graph. Nonetheless, there is increasing interest in inferring more complex structures, such as communities, for multiple reasons, including more effective information retrieval and better interpretability. Stochastic blockmodels offer a powerful tool to detect such structure in a network. We thus propose to exploit advances in random graph theory and embed them within the graphical models framework. A consequence of this approach is the propagation of the uncertainty in graph estimation to large-scale structure learning. We consider Bayesian nonparametric stochastic blockmodels as priors on the graph. We extend such models to consider clique-based blocks and to multiple graph settings introducing a novel prior process based on a Dependent Dirichlet process. Moreover, we devise a tailored computation strategy of Bayes factors for block structure based on the Savage-Dickey ratio to test for presence of larger structure in a graph. We demonstrate our approach in simulations as well as on real data applications in finance and transcriptomics.

Mixformer: An improved self-attention architecture applied to multivariate chaotic time series prediction

Multivariate chaotic time series prediction has been a challenging problem, especially the coupling relationship between multiple variables need to be carefully considered. We propose a self-attention architecture with an information interaction module, called mixformer, applied to the multivariate chaotic time series prediction task. First, Based on rethinking the multivariate data structure, we compensate the deficiency of phase space reconstruction with a group of cross-convolution operator that can automatically update parameters, and propose an explicitly designed feature reconstruction module. Then, to address the problem of interactive fusion of series information and channel information, we propose an information interaction module that enables the feature communication by expanding and contracting dimensions. By breaking the communication barrier between series information and channel information, the feature representation capability is enhanced. Finally, we construct mixformer that combines locally sparse features and global context features. And notably, mixformer integrates the information interaction module and the feature reconstruction module to form a continuous solution. By comparing the results with existing models using multiple simulated data (Lorenz, Chen, and Rossler systems) and real-world application (power consumption), it is verified that our proposed model achieves surprising performance in practice.

Principal Component Analysis Technique for Finding the Best Applicant for a Job

This paper focuses on the use of principal component analysis technique (PCA) in choosing the best applicant for a job in Cihan University-Erbil. Cihan University has a panel of judges (University staff) to help in choosing the applicants for a job by evaluating or rating each one on different scale of preference and different type of characteristics. This process usually creates complicated multivariate data structure, which consists of 25 applicants for a job rated by a panel of judges on 17 characteristics [25 rows, applicants, and 17 columns, characteristics]. PCA plays a crucial role in conducting impactful research as it offers a potent technique for analyzing multivariate data. Researchers can utilize this method to extract valuable information that aids decision-makers in problem-solving. To ensure the appropriateness of data for PCA, certain testing procedures are necessary. In this study, two tests, namely the Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy and Bartlett's Test of Sphericity, were performed, and their significance is vital. The findings indicate that the data employed in this research are suitable for PCA. Scoring and ranking procedures as extra tools were used to see that applicant No. (1) is the first accepted for a job, applicant No. (17) is the second, applicant No. (12) is the third, and so on.

Using ACER ConQuest program to examine multidimensional and many-facet models

The main aim of this study was to introduce the ConQuest program, which is used in the analysis of multivariate and multidimensional data structures, and to show its applications on example data structures. To achieve this goal, a basic research approach was applied. Thus, how to use the ConQuest program and how to prepare the data set for analysis were explained step by step. Then, two example applications were made considering the multidimensional structures. Finally, different sources of variability (e.g., item, student, rater, gender), which are both multidimensional and independent of each other, were performed by considering different sources of variability together. According to the analyses, the dimensionality of the data structures must be examined in the analysis process. If the data structure is multidimensional, appropriate multidimensional IRT analyses should be performed.

Multivariate outlier explanations using Shapley values and Mahalanobis distances

For the purpose of explaining multivariate outlyingness, it is shown that the squared Mahalanobis distance of an observation can be decomposed into outlyingness contributions originating from single variables. The decomposition is obtained using the Shapley value, a well-known concept from game theory that became popular in the context of Explainable AI. In addition to outlier explanation, this concept also relates to the recent formulation of cellwise outlyingness, where Shapley values can be employed to obtain variable contributions for outlying observations with respect to their “expected” position given the multivariate data structure. In combination with squared Mahalanobis distances, Shapley values can be calculated at a low numerical cost, making them an even more attractive tool for outlier interpretation. Simulations and real-world data examples demonstrate the usefulness of these concepts.

Development of a Distribution Free Multivariate Canonical Analysis

This study illustrates the use of Median Polish analysis (MP) as a distribution free procedure that can be used to identify multivariate canonical data structures. The MP may be especially useful in situations where the sample sizes are small, or where the distributions do not meet the assumptions of conventional Canonical Correlation analysis (CC). We begin by comparing the CC and MP analyses with a sample multivariate data set. We go on to compare Type 1 error rates for each of these analyses using Monte Carlo procedures in which we manipulated sample size and skewness of the data distributions. Results indicated that Type 1 error was significantly higher for the CC relative to the MP when the distributions were skewed and/or when the sample sizes were n=20 or 30.

Rapid Response System Based on Graph Attention Network for Predicting In-Hospital Clinical Deterioration

In-hospital clinical deterioration is a major worldwide healthcare burden in the intensive care units (ICUs), as it requires rapid intervention. Rapid response systems (RRSs) are widely used in many hospitals for the early detection of clinical deterioration to prevent cardiac arrest. Recently, with the increasing use of deep learning (DL) and electronic health records (EHR), many DL models have been developed for the intensive care domain, such as prediction of cardiac arrest, sepsis, or transferring to ICU. However, most existing methods do not explicitly learn the structure of multivariate time-series data, and this leads to high false-alarm rates and low sensitivity. In this research, we propose a novel DL-based framework that interpolates high-dimensional sequential data. Our approach combines two graph neural networks with an attention mechanism to learn the complex dependencies among multivariate time series. The experiments were conducted on two datasets: a private clinical dataset collected from Chonnam National University Hospital (CNUH) and a public dataset from the University of Virginia (UV). The experimental results show the potential performance of our model compared to some other related research.

Infants’ gut microbiome data: A Bayesian Marginal Zero-inflated Negative Binomial regression model for multivariate analyses of count data

The infants' gut microbiome is dynamic in nature. Literature has shown high inter-individual variability of gut microbial composition in the early years of infancy compared to adulthood. Although next-generation sequencing technologies are rapidly evolving, several statistical analysis aspects need to be addressed to capture the variability and dynamic nature of the infants' gut microbiome. In this study, we proposed a Bayesian Marginal Zero-inflated Negative Binomial (BAMZINB) model, addressing complexities associated with zero-inflation and multivariate structure of the infants' gut microbiome data. Here, we simulated 32 scenarios to compare the performance of BAMZINB with glmFit and BhGLM as the two other widely similar methods in the literature in handling zero-inflation, over-dispersion, and multivariate structure of the infants' gut microbiome. Then, we showed the performance of the BAMZINB approach on a real dataset using SKOT cohort (I and II) studies. Our simulation results showed that the BAMZINB model performed as well as those two methods in estimating the average abundance difference and had a better fit for almost all scenarios when the signal and sample size were large. Applying BAMZINB on SKOT cohorts showed remarkable changes in the average absolute abundance of specific bacteria from 9 to 18 months for infants of healthy and obese mothers. In conclusion, we recommend using the BAMZINB approach for infants' gut microbiome data taking zero-inflation and over-dispersion properties into account in multivariate analysis when comparing the average abundance difference.

Estimating a panel MSK dataset for comparative analyses of national absorptive capacity systems, economic growth, and development in low and middle income countries.

Within the national innovation system literature, the low- and middle-income countries (LMICs) eligible for the World Bank's International Development Association (IDA) support, are rarely part of empirical discourses on growth, development, and innovation. One major issue hindering empirical analyses in LMICs is the lack of complete data availability. This work offers a new full panel dataset with no missing values for IDA-eligible LMICs. I use a standard, widely respected multiple imputation method (specifically, Predictive Mean Matching) developed by Rubin in the 1980s, which conforms to the structure of multivariate continuous panel data at the country level. The incomplete input data consisting of many variables come from publicly available established sources. These variables, in turn, capture six crucial country-level capacities: technological capacity, financial capacity, human capital capacity, infrastructural capacity, public policy capacity, and social capacity. Such capacities are part and parcel of the National Absorptive Capacity Systems (NACS). The dataset (MSK dataset) thus produced contains data on 47 variables for 82 LMICs between 2005 and 2019. The dataset has passed a quality and reliability check and can therefore be used for comparative analyses of national absorptive capacities and development, transition, and convergence among LMICs.

Dynamic clustering of multivariate panel data

We propose a dynamic clustering model for uncovering latent time-varying group structures in multivariate panel data. The model is dynamic in three ways. First, the cluster location and scale matrices are time-varying to track gradual changes in cluster characteristics over time. Second, all units can transition between clusters based on a Hidden Markov model (HMM). Finally, the HMM’s transition matrix can depend on lagged time-varying cluster distances as well as economic covariates. Monte Carlo experiments suggest that the units can be classified reliably in a variety of challenging settings. Incorporating dynamics in the cluster composition proves empirically important in a study of 299 European banks between 2008Q1 and 2018Q2. We find that approximately 3% of banks transition per quarter on average. Transition probabilities are in part explained by differences in bank profitability, suggesting that factors contributing to low profitability for some banks can lead to long-lasting changes in financial industry structure.

Modal clustering on PPGMMGA projection subspace

SummaryPPGMMGA is a projection pursuit (PP) algorithm aimed at detecting and visualising clustering structures in multivariate data. The algorithm uses the negentropy as PP index obtained by fitting Gaussian mixture models (GMMs) for density estimation and, then, exploits genetic algorithms (GAs) for its optimisation. Since the PPGMMGA algorithm is a dimension reduction technique specifically introduced for visualisation purposes, cluster memberships are not explicitly provided. In this paper a modal clustering approach is proposed for estimating clusters of projected data points. In particular, a modal EM algorithm is employed to estimate the modes corresponding to the local maxima in the projection subspace of the underlying density estimated using parsimonious GMMs. Data points are then clustered according to the domain of attraction of the identified modes. Simulated and real data are discussed to illustrate the proposed method and evaluate the clustering performance.

Hypergraph regularized semi-supervised support vector machine

At present, graph regularized semi-supervised methods achieve excellent performance in various fields. However, the manifold regularization term of most methods only considers the pairwise relationship between data, thus it cannot accurately represent the multivariate and complex structure of data. In this paper, we exploit the multivariate manifold structure by hypergraph, and propose a hypergraph regularized semi-supervised support vector machine (HGSVM) algorithm. To accelerate the training process of HGSVM, we further develop a fast algorithm based boundary sample selection algorithm, termed fast-HGSVM. Moreover, two SMOTE-variant techniques and the one-vs-rest strategy are introduced in fast-HGSVM, and two multi-category semi-supervised algorithms called fast-ASHGSVM and fast-KSHGSVM are proposed. Experiments on two moons and UCI datasets validate the effectiveness of the proposed algorithms.

A new parametric measure of functional dissimilarity: Bridging the gap between the Bray-Curtis dissimilarity and the Euclidean distance

Community ecologists usually consider the Euclidean distance inappropriate to explore the multivariate structure of species abundance data. This is because the Euclidean distance may lead to the counterintuitive result for which two sample plots with no species in common may be more similar to each other than two plots that share the same species list. To overcome this paradoxical situation, the species abundances need to be normalized in some way. Among the many coefficients used by ecologists for the analysis of assemblage data, the Bray-Curtis dissimilarity is certainly the most commonly used. This measure entails normalization of species-wise differences between two plots by the total species abundance in both plots. By highlighting the relationship between the Bray-Curtis dissimilarity and the Euclidean distance, we propose a parametric dissimilarity measure that is appropriate for handling data on community composition. We also show how the new parametric measure can be generalized to the measurement of functional dissimilarity between two plots. A small dataset on the species functional turnover along a chronosequence on Alpine grasslands is used to illustrate the behavior of the proposed measure.

Bending behaviour of thermoplastic composites in melt: A data-driven approach

The resistance to bending of continuous fibre-reinforced thermoplastic composites at processing temperature is an important predictor of wrinkle formation during the stamp forming process. This resistance can be quantified with experiments and approximated with models. However, current models are validated with one, or at most two, thermoplastic composites. Hence, it is not known whether these models generalise to other thermoplastic composites. This data-driven study proposes a rate- and temperature-dependent bending model derived from the experimental data of 10 very different thermoplastic composites. The multivariate data structure was analysed with a Tucker decomposition, together with bootstrapping and cross-validation, to reveal the important generalisable trends, and discard irrelevant features. The most suitable Tucker model could be parametrised to yield a four-parameter model, which approximates the deformation, rate, and temperature dependence, respectively using a linear, a power law, and an Arrhenius-type relation. The four-parameter model was validated with experimental data from four other thermoplastic composite materials. It can describe the measured rate- and temperature-dependent bending behaviour of a wide range of thermoplastic composites.

Partial separability and functional graphical models for multivariate Gaussian processes

SummaryThe covariance structure of multivariate functional data can be highly complex, especially if the multivariate dimension is large, making extensions of statistical methods for standard multivariate data to the functional data setting challenging. For example, Gaussian graphical models have recently been extended to the setting of multivariate functional data by applying multivariate methods to the coefficients of truncated basis expansions. However, compared with multivariate data, a key difficulty is that the covariance operator is compact and thus not invertible. This paper addresses the general problem of covariance modelling for multivariate functional data, and functional Gaussian graphical models in particular. As a first step, a new notion of separability for the covariance operator of multivariate functional data is proposed, termed partial separability, leading to a novel Karhunen–Loève-type expansion for such data. Next, the partial separability structure is shown to be particularly useful in providing a well-defined functional Gaussian graphical model that can be identified with a sequence of finite-dimensional graphical models, each of identical fixed dimension. This motivates a simple and efficient estimation procedure through application of the joint graphical lasso. Empirical performance of the proposed method for graphical model estimation is assessed through simulation and analysis of functional brain connectivity during a motor task.

Analysis of multivariate longitudinal immuno-epidemiological data using a pairwise joint modelling approach

BackgroundImmuno-epidemiologists are often faced with multivariate outcomes, measured repeatedly over time. Such data are characterised by complex inter- and intra-outcome relationships which must be accounted for during analysis. Scientific questions of interest might include determining the effect of a treatment on the evolution of all outcomes together, or grouping outcomes that change in the same way. Modelling the different outcomes separately may not be appropriate because it ignores the underlying relationships between outcomes. In such situations, a joint modelling strategy is necessary. This paper describes a pairwise joint modelling approach and discusses its benefits over more simple statistical analysis approaches, with application to data from a study of the response to BCG vaccination in the first year of life, conducted in Entebbe, Uganda.MethodsThe study aimed to determine the effect of maternal latent Mycobacterium tuberculosis infection (LTBI) on infant immune response (TNF, IFN-γ, IL-13, IL-10, IL-5, IL-17A and IL-2 responses to PPD), following immunisation with BCG. A simple analysis ignoring the correlation structure of multivariate longitudinal data is first shown. Univariate linear mixed models are then used to describe longitudinal profiles of each outcome, and are then combined into a multivariate mixed model, specifying a joint distribution for the random effects to account for correlations between the multiple outcomes. A pairwise joint modelling approach, where all possible pairs of bivariate mixed models are fitted, is then used to obtain parameter estimates.ResultsUnivariate and pairwise longitudinal analysis approaches are consistent in finding that LTBI had no impact on the evolution of cytokine responses to PPD. Estimates from the pairwise joint modelling approach were more precise. Major advantages of the pairwise approach include the opportunity to test for the effect of LTBI on the joint evolution of all, or groups of, outcomes and the ability to estimate association structures of the outcomes.ConclusionsThe pairwise joint modelling approach reduces the complexity of analysis of high-dimensional multivariate repeated measures, allows for proper accounting for association structures and can improve our understanding and interpretation of longitudinal immuno-epidemiological data.

Multivariate Generalized Linear Mixed-Effects Models for the Analysis of Clinical Trial-Based Cost-Effectiveness Data.

Economic evaluations conducted alongside randomized controlled trials are a popular vehicle for generating high-quality evidence on the incremental cost-effectiveness of competing health care interventions. Typically, in these studies, resource use (and by extension, economic costs) and clinical (or preference-based health) outcomes data are collected prospectively for trial participants to estimate the joint distribution of incremental costs and incremental benefits associated with the intervention. In this article, we extend the generalized linear mixed-model framework to enable simultaneous modeling of multiple outcomes of mixed data types, such as those typically encountered in trial-based economic evaluations, taking into account correlation of outcomes due to repeated measurements on the same individual and other clustering effects. We provide new wrapper functions to estimate the models in Stata and R by maximum and restricted maximum quasi-likelihood and compare the performance of the new routines with alternative implementations across a range of statistical programming packages. Empirical applications using observed and simulated data from clinical trials suggest that the new methods produce broadly similar results as compared with Stata’s merlin and gsem commands and a Bayesian implementation in WinBUGS. We highlight that, although these empirical applications primarily focus on trial-based economic evaluations, the new methods presented can be generalized to other health economic investigations characterized by multivariate hierarchical data structures.

A review of second-order blind identification methods.

Second‐order source separation (SOS) is a data analysis tool which can be used for revealing hidden structures in multivariate time series data or as a tool for dimension reduction. Such methods are nowadays increasingly important as more and more high‐dimensional multivariate time series data are measured in numerous fields of applied science. Dimension reduction is crucial, as modeling such high‐dimensional data with multivariate time series models is often impractical as the number of parameters describing dependencies between the component time series is usually too high. SOS methods have their roots in the signal processing literature, where they were first used to separate source signals from an observed signal mixture. The SOS model assumes that the observed time series (signals) is a linear mixture of latent time series (sources) with uncorrelated components. The methods make use of the second‐order statistics—hence the name “second‐order source separation.” In this review, we discuss the classical SOS methods and their extensions to more complex settings. An example illustrates how SOS can be performed.This article is categorized under:Statistical Models > Time Series ModelsStatistical and Graphical Methods of Data Analysis > Dimension ReductionData: Types and Structure > Time Series, Stochastic Processes, and Functional Data