Models For Multivariate Data Research Articles

Hidden Markov models for multivariate panel data

Hidden Markov models for multivariate panel data

Zero-inflated multivariate tobit regression modeling

A frequent challenge encountered in real-world applications is data having a high proportion of zeros. Focusing on ecological abundance data, much attention has been given to zero-inflated count data. Models for non-negative continuous abundance data with an excess of zeros are rarely discussed. Work presented here considers the creation of a point mass at zero through a left-censoring approach or through a hurdle approach. We incorporate both mechanisms to capture the analog of zero-inflation for count data. Additionally, primary attention has been given to univariate zero-inflated modeling (e.g., single species), whereas data often arise jointly (e.g., a collection of species). With multivariate abundance data, a key issue is to capture dependence among the species at a site, both in terms of positive abundance as well as absence. Therefore, our contribution is a model for multivariate zero-inflated continuous data that are non-negative. Working in a Bayesian framework, we discuss the issue of separating the two sources of zeros and offer model comparison metrics for multivariate zero-inflated data. In an application, we model the total biomass for five tree species obtained from plots established in the Forest Inventory Analysis database in the Northeast region of the United States.

Bayesian joint modeling of high-dimensional discrete multivariate longitudinal data using generalized linear mixed models

Bayesian joint modeling of high-dimensional discrete multivariate longitudinal data using generalized linear mixed models

Joint mixed membership modeling of multivariate longitudinal and survival data for learning the individualized disease progression

Joint mixed membership modeling of multivariate longitudinal and survival data for learning the individualized disease progression

Using matrix assisted laser desorption ionisation mass spectrometry combined with machine learning for vaccine authenticity screening

The global population is increasingly reliant on vaccines to maintain population health with billions of doses used annually in immunisation programmes. Substandard and falsified vaccines are becoming more prevalent, caused by both the degradation of authentic vaccines but also deliberately falsified vaccine products. These threaten public health, and the increase in vaccine falsification is now a major concern. There is currently no coordinated global infrastructure or screening methods to monitor vaccine supply chains. In this study, we developed and validated a matrix-assisted laser desorption/ionisation-mass spectrometry (MALDI-MS) workflow that used open-source machine learning and statistical analysis to distinguish authentic and falsified vaccines. We validated the method on two different MALDI-MS instruments used worldwide for clinical applications. Our results show that multivariate data modelling and diagnostic mass spectra can be used to distinguish authentic and falsified vaccines providing proof-of-concept that MALDI-MS can be used as a screening tool to monitor vaccine supply chains.

Parametric and semiparametric approaches for copula-based regression estimation

Based on the normality assumption on dependent variable, regression analysis is one of the most popular statistical techniques for studying the dependence between response and explanatory variables. However, violation of this assumption in the data makes regression analysis inappropriate in several real life situations. Copula is a powerful tool for modeling multivariate data and have recently been employed in regression analysis. The key concept behind copula-based regression approach is to formulate conditional expectation in terms of copula density and marginal distributions. In this paper, we explore parametric and semiparametric estimations of the copula-based regression function. The maximum likelihood (ML), inference functions for margins (IFM), and pseudo maximum likelihood (PML) techniques are adopted here for estimation purposes. Extensive numerical experiments are performed to illustrate the performance of the proposed copula-based regression estimators under specified and misspecified scenarios of copulas and marginals. Finally, two real data applications are also presented to demonstrate the performance of the considered estimators.

Mapping socio-economic status using mixed data: a hierarchical Bayesian approach

Abstract We propose a Bayesian hierarchical model to estimate a socio-economic status (SES) index based on mixed dichotomous and continuous variables. In particular, we extend Quinn’s ([2004]. Bayesian factor analysis for mixed ordinal and continuous responses. Political Analysis, 12(4), 338–353. https://doi.org/10.1093/pan/mph022) and Schliep and Hoeting’s ([2013]. Multilevel latent Gaussian process model for mixed discrete and continuous multivariate response data. Journal of Agricultural, Biological, and Environmental Statistics, 18(4), 492–513. https://doi.org/10.1007/s13253-013-0136-z) factor analysis models for mixed dichotomous and continuous variables by allowing a spatial hierarchical structure of key parameters of the model. Unlike most SES assessment models proposed in the literature, the hierarchical nature of this model enables the use of census observations at the household level without needing to aggregate any information a priori. Therefore, it better accommodates the variability of the SES between census tracts and the number of households per area. The proposed model is used in the estimation of a socio-economic index using 10% of the 2010 Ghana census in the Greater Accra Metropolitan area. Out of the 20 observed variables, the number of people per room, access to water piping and flushable toilets differentiated high and low SES areas the best.

SGG-DGCN: Wind Turbine Anomaly Identification by Using Deep Graph Convolutional Networks with Similarity Graph Generation Strategy

In order to minimize wind turbine failures, fault diagnosis of wind turbines is becoming increasingly important, deep learning methods excel at multivariate monitoring and data modeling, but they are often limited to Euclidean space and struggle to capture the complex coupling between wind turbine sensors. To address this problem, we convert SCADA data into graph data, where sensors act as nodes and their topological connections act as edges, to represent these complex relationships more efficiently. Specifically, a wind turbine anomaly identification method based on deep graph convolutional neural network using similarity graph generation strategy (SGG-DGCN) is proposed. Firstly, a plurality of similarity graphs containing similarity information between nodes are generated by different distance metrics. Then, the generated similarity graphs are fused using the proposed similarity graph generation strategy. Finally, the fused similarity graphs are fed into the DGCN model for anomaly identification. To verify the effectiveness of the proposed SGG-DGCN model, we conducted a large number of experiments. The experimental results show that the proposed SGG-DGCN model has the highest accuracy compared with other models. In addition, the results of ablation experiment also demonstrate that the proposed SGG strategy can effectively improve the accuracy of WT anomaly identification. Conflict of Interest Statement The authors declare no conflicts of interest.

Flexible and tractable modeling of multivariate data using composite Bayesian networks

Flexible and tractable modeling of multivariate data using composite Bayesian networks

Rapid Detection of Cleanliness on Direct Bonded Copper Substrate by Using UV Hyperspectral Imaging.

In the manufacturing process of electrical devices, ensuring the cleanliness of technical surfaces, such as direct bonded copper substrates, is crucial. An in-line monitoring system for quality checking must provide sufficiently resolved lateral data in a short time. UV hyperspectral imaging is a promising in-line method for rapid, contactless, and large-scale detection of contamination; thus, UV hyperspectral imaging (225-400 nm) was utilized to characterize the cleanliness of direct bonded copper in a non-destructive way. In total, 11 levels of cleanliness were prepared, and a total of 44 samples were measured to develop multivariate models for characterizing and predicting the cleanliness levels. The setup included a pushbroom imager, a deuterium lamp, and a conveyor belt for laterally resolved measurements of copper surfaces. A principal component analysis (PCA) model effectively differentiated among the sample types based on the first two principal components with approximately 100.0% explained variance. A partial least squares regression (PLS-R) model to determine the optimal sonication time showed reliable performance, with R2cv = 0.928 and RMSECV = 0.849. This model was able to predict the cleanliness of each pixel in a testing sample set, exemplifying a step in the manufacturing process of direct bonded copper substrates. Combined with multivariate data modeling, the in-line UV prototype system demonstrates a significant potential for further advancement towards its application in real-world, large-scale processes.

A Bayesian non-stationary heteroskedastic time series model for multivariate critical care data.

We propose a multivariate GARCH model for non-stationary health time series by modifying the observation-level variance of the standard state space model. The proposed model provides an intuitive and novel way of dealing with heteroskedastic data using the conditional nature of state-space models. We follow the Bayesian paradigm to perform the inference procedure. In particular, we use Markov chain Monte Carlo methods to obtain samples from the resultant posterior distribution. We use the forward filtering backward sampling algorithm to efficiently obtain samples from the posterior distribution of the latent state. The proposed model also handles missing data in a fully Bayesian fashion. We validate our model on synthetic data and analyze a data set obtained from an intensive care unit in a Montreal hospital and the MIMIC dataset. We further show that our proposed models offer better performance, in terms of WAIC than standard state space models. The proposed model provides a new way to model multivariate heteroskedastic non-stationary time series data. Model comparison can then be easily performed using the WAIC.

A common feature-driven prediction model for multivariate time series data

Multivariate time series data contain a variety of common features that are difficult to extract, among which the sudden irregular fluctuation trend, the trend feature of large fluctuation amplitude, and the long-term dependency relationship in the time series have an important impact on the accuracy of the prediction model. To predict non-stationary trends in large-scale data accurately using the common characteristics of multivariate time series. A novel generalised forecasting model NeA3L based on common features of time series is developed. The NeA3L model utilizes multiple independent parallel feature extraction modules to obtain the common features of multivariate time series. It utilizes the three-layer iterative structure to deal with sudden irregular fluctuation patterns. NeA3L optimizes the network structure to realize the heterogeneous codec structure. It applies the attention mechanism between the encoder and the decoder to accomplish the multivariate prediction of the multivariate time series, which has good stability for predicting various types of multivariate data. A comparison of the NeA3L model with nine current time series prediction methods on four publicly available datasets shows that the NeA3L model outperforms the current methods in various evaluation metrics. The RMSE is improved by 2.3 %, 26.5 %, 28.9 %, and 20.84 % on average, respectively. The NeA3L model can be universally applicable to the field of multivariate time series prediction, which is important for optimizing the intelligent management and decision-making.

SERT: A transformer based model for multivariate temporal sensor data with missing values for environmental monitoring

Environmental monitoring is crucial to our understanding of climate change, biodiversity loss and pollution. The availability of large-scale spatio-temporal data from sources such as sensors and satellites allows us to develop sophisticated models for forecasting and understanding key drivers. However, the data collected from sensors often contain missing values due to faulty equipment or maintenance issues. The missing values rarely occur simultaneously leading to data that are multivariate misaligned sparse time series. We propose two models that are capable of performing multivariate spatio-temporal forecasting while handling missing data naturally without the need for imputation. The first model is a transformer-based model, which we name SERT (Spatio-temporal Encoder Representations from Transformers). The second is a simpler model named SST-ANN (Sparse Spatio-Temporal Artificial Neural Network) which is capable of providing interpretable results. We conduct extensive experiments on two different datasets for multivariate spatio-temporal forecasting and show that our models have competitive or superior performance to those at the state-of-the-art.

Manufacturing process transfer to a 30 kg/h continuous direct compression line with real-time composition monitoring

Transferring an existing marketed pharmaceutical product from batch to continuous manufacturing (CM) without changes in regulatory registration is a challenging task in the pharmaceutical industry. Continuous manufacturing can provide an increased production rate and better equipment utilisation while retaining key quality attributes of the final product. Continuous manufacturing necessitates the monitoring of critical quality attributes in real time by appropriate process analytical tools such as near infra-red (NIR) probes. The present work reports a successful transfer of an existing drug product from batch to continuous manufacturing process without changing the formulation. A key step was continuous powder blending, whose design and operating parameters including weir type, agitation rate, dynamic hold-up and residence time were systematically investigated with respect to process repeatability. A NIR-based multivariate data model for in-line composition monitoring has been developed and validated against an existing quality control method for measuring tablet content uniformity. A continuous manufacturing long-run with a throughput of 30 kg/h (approx. 128,000 tablets per hour), uninterrupted for 320 min, has been performed to test and validate the multivariate data model as well as the batch to continuous process transfer. The final disintegration and dissolution properties of tablets manufactured by the continuous process were found to be equivalent to those manufactured by the original batch process.

Testing many constraints in possibly irregular models using incomplete U-statistics

Abstract We consider the problem of testing a null hypothesis defined by equality and inequality constraints on a statistical parameter. Testing such hypotheses can be challenging because the number of relevant constraints may be on the same order or even larger than the number of observed samples. Moreover, standard distributional approximations may be invalid due to irregularities in the null hypothesis. We propose a general testing methodology that aims to circumvent these difficulties. The constraints are estimated by incomplete U-statistics, and we derive critical values by Gaussian multiplier bootstrap. We show that the bootstrap approximation of incomplete U-statistics is valid for kernels that we call mixed degenerate when the number of combinations used to compute the incomplete U-statistic is of the same order as the sample size. It follows that our test controls type I error even in irregular settings. Furthermore, the bootstrap approximation covers high-dimensional settings making our testing strategy applicable for problems with many constraints. The methodology is applicable, in particular, when the constraints to be tested are polynomials in U-estimable parameters. As an application, we consider goodness-of-fit tests of latent-tree models for multivariate data.

A Spectral Method for Identifiable Grade of Membership Analysis with Binary Responses.

Grade of membership (GoM) models are popular individual-level mixture models for multivariate categorical data. GoM allows each subject to have mixed memberships in multiple extreme latent profiles. Therefore, GoM models have a richer modeling capacity than latent class models that restrict each subject to belong to a single profile. The flexibility of GoM comes at the cost of more challenging identifiability and estimation problems. In this work, we propose a singular value decomposition (SVD)-based spectral approach to GoM analysis with multivariate binary responses. Our approach hinges on the observation that the expectation of the data matrix has a low-rank decomposition under a GoM model. For identifiability, we develop sufficient and almost necessary conditions for a notion of expectation identifiability. For estimation, we extract only a few leading singular vectors of the observed data matrix and exploit the simplex geometry of these vectors to estimate the mixed membership scores and other parameters. We also establish the consistency of our estimator in the double-asymptotic regime where both the number of subjects and the number of items grow to infinity. Our spectral method has a huge computational advantage over Bayesian or likelihood-based methods and is scalable to large-scale and high-dimensional data. Extensive simulation studies demonstrate the superior efficiency and accuracy of our method. We also illustrate our method by applying it to a personality test dataset.

Multivariate probit linear mixed models for multivariate longitudinal binary data.

When analyzing multivariate longitudinal binary data, we estimate the effects on the responses of the covariates while accounting for three types of complex correlations present in the data. These include the correlations within separate responses over time, cross-correlations between different responses at different times, and correlations between different responses at each time point. The number of parameters thus increases quadratically with the dimension of the correlation matrix, making parameter estimation difficult; the estimated correlation matrix must also meet the positive definiteness constraint. The correlation matrix may additionally be heteroscedastic; however, the matrix structure is commonly considered to be homoscedastic and constrained, such as exchangeable or autoregressive with order one. These assumptions are overly strong, resulting in skewed estimates of the covariate effects on the responses. Hence, we propose probit linear mixed models for multivariate longitudinal binary data, where the correlation matrix is estimated using hypersphere decomposition instead of the strong assumptions noted above. Simulations and real examples are used to demonstrate the proposed methods. An open source R package, BayesMGLM, is made available on GitHub at https://github.com/kuojunglee/BayesMGLM/ with full documentation to produce the results.

Homogeneity pursuit and variable selection in regression models for multivariate abundance data.

When building regression models for multivariate abundance data in ecology, it is important to allow for the fact that the species are correlated with each other. Moreover, there is often evidence species exhibit some degree of homogeneity in their responses to each environmental predictor, and that most species are informed by only a subset of predictors. We propose a generalized estimating equation(GEE) approach for simultaneous homogeneity pursuit (ie, grouping species with similar coefficient values while allowing differing groups for different covariates) and variable selection in regression models for multivariate abundance data. Using GEEs allows us to straightforwardly account for between-response correlations through a (reduced-rank) working correlation matrix. We augment the GEE with both adaptive fused lasso- and adaptive lasso-type penalties, which aim to cluster the species-specific coefficients within each covariate and encourage differing levels of sparsity across the covariates, respectively. Numerical studies demonstrate the strong finite sample performance of the proposed method relative to several existing approaches for modeling multivariate abundance data. Applying the proposed method to presence-absence records collected along the Great Barrier Reef in Australia reveals both a substantial degree of homogeneity and sparsity in species-environmental relationships. We show this leads to a more parsimonious model for understanding the environmental drivers of seabed biodiversity, and results in stronger out-of-sample predictive performance relative to methods that do not accommodate such features.

Proportional rates models for multivariate panel count data.

Multivariate panel count data arise when there are multiple types of recurrent events, and the observation for each study subject consists of the number of recurrent events of each type between two successive examinations. We formulate the effects of potentially time-dependent covariates on multiple types of recurrent events through proportional rates models, while leaving the dependence structures of the related recurrent events completely unspecified. We employ nonparametric maximum pseudo-likelihood estimation under the working assumptions that all types of events are independent and each type of event is a nonhomogeneous Poisson process, and we develop a simple and stable EM-type algorithm. We show that the resulting estimators of the regression parameters are consistent and asymptotically normal, with a covariance matrix that can be estimated consistently by a sandwich estimator. In addition, we develop a class of graphical and numerical methods for checking the adequacy of the fitted model. Finally, we evaluate the performance of the proposed methods through simulation studies and analysis of a skin cancer clinical trial.

Regression analysis of multivariate recurrent event data allowing time-varying dependence with application to stroke registry data.

In multivariate recurrent event data, each patient may repeatedly experience more than one type of event. Analysis of such data gets further complicated by the time-varying dependence structure among different types of recurrent events. The available literature regarding the joint modeling of multivariate recurrent events assumes a constant dependency over time, which is strict and often violated in practice. To close the knowledge gap, we propose a class of flexible shared random effects models for multivariate recurrent event data that allow for time-varying dependence to adequately capture complex correlation structures among different types of recurrent events. We developed an expectation-maximization algorithm for stable and efficient model fitting. Extensive simulation studies demonstrated that the estimators of the proposed approach have satisfactory finite sample performance. We applied the proposed model and the estimating method to data from a cohort of stroke patients identified in the University of Texas Houston Stroke Registry and evaluated the effects of risk factors and the dependence structure of different types of post-stroke readmission events.