Class Of Latent Variable Models Research Articles

GLAMLE: inference for multiview network data in the presence of latent variables, with an application to commodities trading

The statistical analysis of multiview network data provides researchers with insights on the mechanism governing many complex systems of interconnected elements. A graph is routinely applied to model this type of data: this poses several statistical challenges. Among them, there is the need for considering that some unobservable factors may determine the network connections and the graph topology. This requirement entails both complex models and computationally heavy inference procedures. To cope with these inference issues, a novel and flexible class of latent variable models labeled Graph Generalized Linear Latent Variable Model (GGLLVM) is introduced. Then, its parameters are estimated via the maximization of the Laplace-approximated likelihood. This leads to the creation of an M-estimator which is named Graph Laplace-Approximated Maximum Likelihood Estimator (GLAMLE). The statistical properties of this new estimator are characterized, and numerical experiments on simulated networks are performed to illustrate that the GLAMLE yields fast and accurate inference, in comparison to other widely used approaches. A real data application to commodities trading in European countries illustrates how the use of GGLLVM and GLAMLE unveils the import/export propensity that each node of the network has toward other nodes, along with additional information specific to each traded commodity.

A Note on Hidden Markov Models with Application to Criminal Intelligence

Hidden Markov Models (HMMs) which fall under the class of latent variable models have received widespread attention in many fields of applications. HMMs were initially developed and applied within the context of speech recognition. The theoretical framework underpinning the formalism of HMMs has also evolved over time and has found an exalted place in the theory of stochastic processes. The three problems HMMs are used to resolve were discussed alongside their solutions in this paper. An application to criminal intelligence in unraveling the culprit in a situation involving theft was also carried out and results obtained indicated that the HMMs approach offered a similar result with that of the well-established Dynamic Programming approach.

InfoCEVAE: treatment effect estimation with hidden confounding variables matching

Treatment effect estimation is a fundamental problem in various domains for effective decision making. While many studies assume that observational data include all the confounding variables, we cannot practically guarantee that observational data include such confounding variables, and there might be confounding variables that are not included in observational data, referred to as hidden confounding variables. Recently, variational autencoder (VAE) based methods have been successfully applied to treatment effect estimation problem. However, although they can recover a large class of latent variable models, they do not give the correct treatment effect, even when they achieve an optimal solution due to the nature of VAE loss function. We propose an efficient VAE-based method that employs information theory to estimate treatment effect and combines it with a matching technique. To the best of our knowledge, this is the first work that gives the correct treatment effect given an optimal solution using VAE-based methods. Experiments on a semi-real dataset and synthetic dataset demonstrate that the proposed method mitigates VAE problems and observational bias effectively, even under hidden confounding variables, and outperforms strong baseline methods.

A projection‐based Laplace approximation for spatial latent variable models

AbstractLaplace method is a practical tool for obtaining maximum likelihood estimators for a wide class of latent variable models. The main idea is to approximate the integrand using a Gaussian distribution. However, with increasing observations, the Laplace approximation becomes infeasible because the dimension of the correlated latent variables grows, which results in the high‐dimensional optimization problem. One important example is spatial latent variable models, which are widely used in many fields, such as ecology, epidemiology, and sociology. Spatial latent variable models are useful for investigating the relationship between spatial covariates or predicting the unobserved area. Here, we propose a fast Laplace approximation based on the dimension reduction of the latent variables. Our methods are faster and have fewer components to be tuned than simulation‐based methods such as Markov chain Monte Carlo maximum likelihood and Monte Carlo expectation‐maximization. Our approach can be applied to the large non‐Gaussian spatial data sets, commonly used in modern environmental sciences. Especially, we show how we may understand spatial patterns of non‐Gaussian responses for two case studies: confirmed COVID‐19 cases in the United States and thickness of the Antarctic ice sheet. Through simulation studies under different scenarios, we investigate that our method can provide accurate maximum likelihood estimations and predictions quickly. Our study can be widely applicable for practical maximum likelihood inference for high‐dimensional random effect models. We provide a freely available R‐package that can implement the proposed method.

Item response theory and its applications in educational measurement Part II: Theory and practices of test equating in item response theory

AbstractItem response theory (IRT) is a class of latent variable models, which are used to develop educational and psychological tests (e.g., standardized tests, personality tests, tests for licensure and certification). We offer readers with comprehensive overviews of the theory and applications of IRT through two articles. While Part 1 of the review discusses topics such as foundations of educational measurement, IRT models, item parameter estimation, and applications of IRT with R, this Part 2 reviews areas of test scores based on IRT. The primary focus is on presenting various topics with respect to test equating such as equating designs, IRT‐based equating methods, anchor stability check methods, and impact data analysis that psychometricians would deal with for a large‐scale standardized assessment in practice. These analyses are illustrated in Example section using data from Kolen and Brennan (2014). We also cover the foundation of IRT, IRT‐based person ability parameter estimation methods, and scaling and scale score.This article is categorized under: Applications of Computational Statistics > Psychometrics Software for Computational Statistics > Software/Statistical Software

Statistical Analysis of Multi-Relational Network Recovery

In this paper, we develop asymptotic theories for a class of latent variable models for large-scale multi-relational networks. In particular, we establish consistency results and asymptotic error bounds for the (penalized) maximum likelihood estimators when the size of the network tends to infinity. The basic technique is to develop a non-asymptotic error bound for the maximum likelihood estimators through large deviations analysis of random fields. We also show that these estimators are nearly optimal in terms of minimax risk.

Item response theory and its applications in educational measurement Part I: Item response theory and its implementation in R

AbstractItem response theory (IRT) is a class of latent variable models, which are used to develop educational and psychological tests (e.g., standardized tests, personality tests, tests for licensure, and certification). We review the theory and practices of IRT across two articles. In Part 1, we provide a broad range of topics such as foundations of educational measurement, basics of IRT, and applications of IRT using R. We focus particularly on the topics that the mirt package covers. These include unidimensional and multidimensional IRT models for dichotomous and polytomous items with continuous and discrete factors, confirmatory analysis and multigroup analysis in IRT, and estimation algorithms. In Part 2, on the other hand, we focus on more practical aspects of IRT, namely scoring, scaling, and equating.This article is categorized under: Applications of Computational Statistics > Psychometrics Software for Computational Statistics > Software/Statistical Software

Likelihood-based tests for a class of misspecified finite mixture models for ordinal categorical data

The main purpose of this paper is to apply likelihood-based hypothesis testing procedures to a class of latent variable models for ordinal responses that allow for uncertain answers (Colombi et al. in Scand J Stat, 2018. https://doi.org/10.1111/sjos.12366). As these models are based on some assumptions, needed to describe different respondent behaviors, it is essential to discuss inferential issues without assuming that the tested model is correctly specified. By adapting the works of White (Econometrica 50(1):1–25, 1982) and Vuong (Econometrica 57(2):307–333, 1989), we are able to compare nested models under misspecification and then contrast the limiting distributions of Wald, Lagrange multiplier/score and likelihood ratio statistics with the classical asymptotic Chi-square to show the consequences of ignoring misspecification.

Tensor decompositions for learning latent variable models

This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models--including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation--which exploits a certain tensor structure in their low-order observable moments (typically, of second- and third-order). Specifically, parameter estimation is reduced to the problem of extracting a certain (orthogonal) decomposition of a symmetric tensor derived from the moments; this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches (similar to the case of matrices). A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedin's perturbation theorem for the singular vectors of matrices. This implies a robust and computationally tractable estimation approach for several popular latent variable models.

Bayesian nonhomogeneous Markov models via Pólya-Gamma data augmentation with applications to rainfall modeling

Discrete-time hidden Markov models are a broadly useful class of latent-variable models with applications in areas such as speech recognition, bioinformatics, and climate data analysis. It is common in practice to introduce temporal nonhomogeneity into such models by making the transition probabilities dependent on time-varying exogenous input variables via a multinomial logistic parametrization. We extend such models to introduce additional nonhomogeneity into the emission distribution using a generalized linear model (GLM), with data augmentation for sampling-based inference. However, the presence of the logistic function in the state transition model significantly complicates parameter inference for the overall model, particularly in a Bayesian context. To address this, we extend the recently-proposed Polya-Gamma data augmentation approach to handle nonhomogeneous hidden Markov models (NHMMs), allowing the development of an efficient Markov chain Monte Carlo (MCMC) sampling scheme. We apply our model and inference scheme to 30 years of daily rainfall in India, leading to a number of insights into rainfall-related phenomena in the region. Our proposed approach allows for fully Bayesian analysis of relatively complex NHMMs on a scale that was not possible with previous methods. Software implementing the methods described in the paper is available via the R package NHMM.

Tensor-Variate Restricted Boltzmann Machines

Restricted Boltzmann Machines (RBMs) are an important class of latent variable models for representing vector data. An under-explored area is multimode data, where each data point is a matrix or a tensor. Standard RBMs applying to such data would require vectorizing matrices and tensors, thus resulting in unnecessarily high dimensionality and at the same time, destroying the inherent higher-order interaction structures. This paper introduces Tensor-variate Restricted Boltzmann Machines (TvRBMs) which generalize RBMs to capture the multiplicative interaction between data modes and the latent variables. TvRBMs are highly compact in that the number of free parameters grows only linear with the number of modes. We demonstrate the capacity of TvRBMs on three real-world applications: handwritten digit classification, face recognition and EEG-based alcoholic diagnosis. The learnt features of the model are more discriminative than the rivals, resulting in better classification performance.

Latent variable models for ordinal data by using the adaptive quadrature approximation

Latent variable models for ordinal data represent a useful tool in different fields of research in which the constructs of interest are not directly observable so that one or more latent variables are required to reduce the complexity of the data. In these cases problems related to the integration of the likelihood function of the model can arise. Indeed analytical solutions do not exist and in presence of several latent variables the most used classical numerical approximation, the Gauss Hermite quadrature, cannot be applied since it requires several quadrature points per dimension in order to obtain quite accurate estimates and hence the computational effort becomes not feasible. Alternative solutions have been proposed in the literature, like the Laplace approximation and the adaptive quadrature. Different studies demonstrated the superiority of the latter method particularly in presence of categorical data. In this work we present a simulation study for evaluating the performance of the adaptive quadrature approximation for a general class of latent variable models for ordinal data under different conditions of study. A real data example is also illustrated.

Inference for structural equation modelling on dependent populations

Latent variable modelling is used widely in applications to economics, social and behavioural sciences. Since the normality-based model fitting procedures are simple and broadly available, and since such procedures are often applied to non-normal data or non-random samples, it is important to investigate the appropriateness of such practice and to suggest simple remedies. This paper addresses these issues for the analysis of multiple populations. For a very general class of latent variable models, a particular parameterisation is used for meaningful and interpretable analysis of several populations. It turns out that under this parameterisation the large sample statistical inferences based on the assumption of normal and independent populations are valid for virtually any non-normal and dependent populations. This result is also valid when some latent variables are treated as fixed instead of random, or when a group of individuals is measured over several time points longitudinally.

A Multivariate Approach for the Analysis of Spatially Correlated Environmental Data

The formulation and the evaluation of environmental policy depend upon a general class of latent variable models known as multivariate receptor models. Estimation of the number of major pollution sources, the source composition profiles and the source contributions are the main interests in multivariate receptor modelling. Many different approaches have been proposed both when the number of sources is unknown (explorative factorial analysis) and when the number and the type of sources are known (regression models). The objective of this work is to propose a flexible approach to the multivariate receptor models that incorporates the extra variability due to the spatial dependence. The method is applied to Lombardia air pollution data.

A general class of latent variable models for ordinal manifest variables with covariate effects on the manifest and latent variables.

Previous work on a general class of multidimensional latent variable models for analysing ordinal manifest variables is extended here to allow for direct covariate effects on the manifest ordinal variables and covariate effects on the latent variables. A full maximum likelihood estimation method is used to estimate all the model parameters simultaneously. Goodness-of-fit statistics and standard errors are discussed. Two examples from the 1996 British Social Attitudes Survey are used to illustrate the methodology.

Robustness of Statistical Inference in Factor Analysis and Related Models

A class of latent variable models which includes the unrestricted factor analysis model is considered. It is shown that minimum discrepancy test statistics and estimators derived under normality assumptions retain their asymptotic properties when the common factors are not normally distributed but the unique factors do have a multivariate normal distribution. The minimum discrepancy test statistics and estimators considered include the usual likelihood ratio test statistic and maximum likelihood estimators.

Robustness of statistical inference in factor analysis and related models

SUMMARY A class of latent variable models which includes the unrestricted factor analysis model is considered. It is shown that minimum discrepancy test statistics and estimators derived under normality assumptions retain their asymptotic properties when the common factors are not normally distributed but the unique factors do have a multivariate normal distribution. The minimum discrepancy test statistics and estimators considered include the usual likelihood ratio test statistic and maximum likelihood estimators.

Conditional Association and Unidimensionality in Monotone Latent Variable Models

Latent variable models represent the joint distribution of observable variables in terms of a simple structure involving unobserved or latent variables, usually assuming the conditional independence of the observable variables given the latent variables. These models play an important role in educational measurement and psychometrics, in sociology and in population genetics, and are implicit in some work on systems reliability. We study a broad class of latent variable models, namely the monotone unidimensional models, in which the latent variable is a scalar, the observable variables are conditionally independent given the latent variable and the conditional distribution of the observables given the latent variable is stochastically increasing in the latent variable. All models in this class imply a new strong form of positive dependence among the observable variables, namely conditional (positive) association. This positive dependence condition may be used to test whether any model in this class can provide an adequate fit to observed data. Various applications, generalizations and a numerical example are discussed.

CONDITIONAL ASSOCIATION AND UNIDIMENSIONALITY IN MONOTONE LATENT VARIABLE MODELS

ABSTRACTLatent variable models represent the joint distribution of observable variables in terms of a simple structure involving unobserved or latent variables, usually assuming the conditional independence of the observable variables given the latent variables. These models play an important role in educational measurement and psychometrics, in sociology, in population genetics, and are implicit in some work on systems reliability. We study a broad class of latent variable models, namely the monotone unidimensional models, in which the latent variable is a scalar, the observable variables are conditionally independent given the latent variable, and the conditional distribution of the observables given the latent variable is stochastically increasing in the latent variable. All models in this class imply a new strong form of positive dependence among the observable variables, namely conditional (positive) association. This positive dependence condition may be used to test whether any model in this class can provide an adequate fit to observed data. Various applications, generalizations, and a numerical example are discussed.

Latent variable models for time series: A frequency domain approach with an application to the permanent income hypothesis

The theory of estimation and inference in a very general class of latent variable models for time series is developed by showing that the distribution theory for the finite Fourier transform of the observable variables in latent variable models for time series is isomorphic to that for the observable variables themselves in classical latent variable models. This implies that analytic work on classical latent variable models can be adapted to latent variable models for time series, an implication which is illustrated here in the context of a general canonical form. To provide an empirical example a latent variable model for permanent income is developed, its parameters are shown to be identified, and a variety of restrictions on these parameters implied by the permanent income hypothesis are tested.