Probabilistic Generative Model Research Articles

Fundamental study on probabilistic generative modeling of earthquake ground motion time histories using generative adversarial networks

AbstractThis study proposes a probabilistic model for earthquake ground motion prediction, named ground motion generation model, which can generate ground motion time history data directly. The ground motion generation model is based on a data‐driven technique called generative adversarial networks, allowing generation of ground motion time history data without making assumptions about physical or statistical models. A method to quantitatively and qualitatively evaluate the performance of constructed model is also proposed and the ground motion generation model is optimized for high performance from earthquake engineering and deep learning perspectives. Numerical experiments show that our proposed model is probabilistic, approximating the probabilistic distribution of the dataset of observed records and generating realistic ground motion time histories with various characteristics in the time and frequency domains.

Gaussian Process Koopman Mode Decomposition.

We propose a nonlinear probabilistic generative model of Koopman mode decomposition based on an unsupervised gaussian process. Existing data-driven methods for Koopman mode decomposition have focused on estimating the quantities specified by Koopman mode decomposition: eigenvalues, eigenfunctions, and modes. Our model enables the simultaneous estimation of these quantities and latent variables governed by an unknown dynamical system. Furthermore, we introduce an efficient strategy to estimate the parameters of our model by low-rank approximations of covariance matrices. Applying the proposed model to both synthetic data and a real-world epidemiological data set, we show that various analyses are available using the estimated parameters.

Influence of undirected learning on improving the accuracy of everyday predictions: an experiment in social metacognition

The article presents the results of a sociological experiment that allowed to take a fresh look at the live issue of how people are able to learn the evaluation of the parameters of the distributions of certain important social variables and assimilate probabilistic generative models of the surrounding reality in the process of implicit non-directed learning that takes place in everyday life. The article shows that the observed effects of episodic learning can be influenced by implicit (background) knowledge about local realities (in this particular case, Russian ones). The authors also found that a number of other factors can also influence the effects of learning: everyday social interactions in a particular region or city, a format for presenting examples that is close to experience. An increase in the accuracy of the forecasts made can also be a consequence of the possibility of extrapolating easily remembered data. The data obtained as a result of the experiment allow us to conclude that the initial "illusion of awareness", discovered earlier and described in the previous works of the authors, is not associated with the metacognitive ability to regulate one's judgments based on the obtained imitation of episodic (situational) "everyday experience”.

Sentiment Analysis of Weibo Comments based on LDA Model

The essence of LDA (Latent Dirichlet Allocation) model is a generative Bayesian probability model that contains three layers of words, topics and corpus (sometimes called document set). Under the LDA algorithm theory, each document represents a probability distribution formed by some topics, and each topic represents a probability distribution formed by many words. Therefore, the model fitting results will present the core keywords and specific probabilities of each topic, and researchers can interpret the meaning of the document according to the model results. In this paper, we hope to use the LDA model as the basis for the emotional analysis of microblog comments.

Learning the progression patterns of treatments using a probabilistic generative model

Modeling a disease or the treatment of a patient has drawn much attention in recent years due to the vast amount of information that Electronic Health Records contain. This paper presents a probabilistic generative model of treatments that are described in terms of sequences of medical activities of variable length. The main objective is to identify distinct subtypes of treatments for a given disease, and discover their development and progression. To this end, the model considers that a sequence of actions has an associated hierarchical structure of latent variables that both classifies the sequences based on their evolution over time, and segments the sequences into different progression stages. The learning procedure of the model is performed with the Expectation–Maximization algorithm which considers the exponential number of configurations of the latent variables and is efficiently solved with a method based on dynamic programming. The evaluation of the model is twofold: first, we use synthetic data to demonstrate that the learning procedure allows the generative model underlying the data to be recovered; we then further assess the potential of our model to provide treatment classification and staging information in real-world data. Our model can be seen as a tool for classification, simulation, data augmentation and missing data imputation.

On Applicability of Quantum Formalism to Model Decision Making: Can Cognitive Signaling Be Compatible with Quantum Theory?

This note is devoted to the problem of signaling (marginal inconsistency) in the Bell-type experiments with physical and cognitive systems. It seems that in quantum physics, this problem is still not taken seriously. Only recently have experimenters started to check the signaling hypothesis for their data. For cognitive systems, signaling was statistically significant in all experiments (typically for decision making) performed up to today. Here, one cannot simply ignore this problem. Since signaling contradicts the quantum theory of measurement for compatible observables, its statistical significance in experiments with humans can be considered as an objection for quantum-like modeling-applications of quantum theory to cognition, decision making, psychology, economics and finance, social and political science. In this paper, we point to two possible sources of signaling generation that are consistent with quantum measurement theory. Thus, the signaling objection for quantum-like modeling is not catastrophic. One of these sources is the direct physical signaling about selection of experimental settings, questions or tasks in quantum-like studies. Another possible source is a state modification dependent on experimental settings. The latter was a rather common source of signaling in quantum physics. Since the physical size of the brain is very small comparing with the light velocity, it seems to be impossible to prevent the direct physical signaling (with electromagnetic waves) between the brain's areas processing two questions a and b. However, if, for these questions, not the electromagnetic waves, but electrochemical communication plays the crucial role, the experimenter may hope to make signaling weaker by answering the questions faster. The problem of question-dependent mental state modification seems to be solvable via smarter experimental design. This paper can be useful both for physicists interested in quantum foundations and for researchers working in quantum-like studies, e.g., applying the quantum theory to model decision making or psychological effects. This paper is solely about quantum theory. Thus, we do not consider general contextual probabilistic models.

A computational model of aesthetic value.

People invest precious time and resources on experiences such as watching movies or listening to music. Yet, we still have a poor understanding of how such sensed experiences gain aesthetic value. We propose a model of aesthetic value that integrates existing theories with literature on conventional primary and secondary rewards such as food and money. We assume that the states of observers' sensory and cognitive systems adapt to process stimuli effectively in both the present and the future. These system states collectively comprise a probabilistic generative model of stimuli in the environment. Two interlinked components generate value: immediate sensory reward and the change in expected future reward. An immediate sensory reward is taken as the fluency with which a stimulus is processed, quantified by the likelihood of that stimulus given an observer's state. The change in expected future reward is taken as the change in fluency with which likely future stimuli will be processed. It is quantified by the change in the divergence between the observer's system state and the distribution of stimuli that the observer expects to see over the long term. Simulations show that a simple version of the model can account for empirical data on the effects of exposure, complexity, and symmetry on aesthetic value judgments. Taken together, our model melds processing fluency theories (immediate reward) and learning theories (change in expected future reward). Its application offers insight as to how the interplay of immediate processing fluency and learning gives rise to aesthetic value judgments. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A stochastic algorithm for solving the posterior inference problem in topic models

Latent Dirichlet allocation (LDA) is an important probabilistic generative model and has usually used in many domains such as text mining, retrieving information, or natural language processing domains. The posterior inference is the important problem in deciding the quality of the LDA model, but it is usually non-deterministic polynomial (NP)-hard and often intractable, especially in the worst case. For individual texts, some proposed methods such as variational Bayesian (VB), collapsed variational Bayesian (CVB), collapsed Gibb’s sampling (CGS), and online maximum a posteriori estimation (OPE) to avoid solving this problem directly, but they usually do not have any guarantee of convergence rate or quality of learned models excepting variants of OPE. Based on OPE and using the Bernoulli distribution combined, we design an algorithm namely general online maximum a posteriori estimation using two stochastic bounds (GOPE2) for solving the posterior inference problem in LDA model. It also is the NP-hard non-convex optimization problem. Via proof of theory and experimental results on the large datasets, we realize that GOPE2 is performed to develop the efficient method for learning topic models from big text collections especially massive/streaming texts, and more efficient than previous methods.

A Hierarchical Bayesian Fusion Method of Infrared and Visible Images for Temperature Monitoring of High-Speed Direct-Drive Blower

The high-speed direct-drive blower is a high-efficiency, energy-saving, and environmentally friendly blower, and it is widely applied in industrial fields. The temperature changes significantly affect the performance of the high-speed direct-drive blower, and it is crucial to monitor its temperature. Infrared and visible light cameras are simultaneously used to measure and characterize temperature information. The measurement method can have both the temperature information of the infrared image and the contrast information of the structure and contour of the visible light image. This article establishes an image fusion representation and probabilistic generation model of infrared and visible images. Then, the information of the infrared and visible images is fused under the Bayesian framework. A hierarchical prior model using the Haar wavelet transform is proposed. The joint maximum a posteriori criterion is chosen, and an appropriate alternate optimization algorithm is designed to achieve information fusion. The proposed method is validated in industrial scenarios of high-speed direct-drive blowers. The experimental results demonstrate the robustness and effectiveness of the proposed method.

Community detection in weighted networks using probabilistic generative model

Community detection in networks is a useful tool for detecting the behavioral and inclinations of users to a specific topic or title. Weighted, unweighted, directed, and undirected networks can all be used for detecting communities depending on the network structure and content. The proposed model framework for community detection is based on weighted networks. We use two important and effective concepts in graph analysis. The structural density between nodes is the first concept, and the second is the weight of edges between nodes. The proposed model advantage is using a probabilistic generative model that estimates the latent parameters of the probabilistic model and detecting the community based on the probability of the presence or absence of weighted edge. The output of the proposed model is the intensity of belonging each weighted node to the communities. A relationship between the observation of a pair of nodes in multiple communities and the probability of an edge with a high weight between them, is one of the important outputs that interpret the detected communities by finding relevancy between membership of nodes to communities and edge weight. Experiments are performed on real-world weighted networks and synthetic weighted networks to evaluate the performance and accuracy of the proposed algorithm. The results will show that the proposed algorithm is more density and accurate than other algorithms in weighted community detection.

Topic Modeling with Transformers for Sentence-Level Using Coronavirus Corpus

A Topic Model is a class of generative probabilistic models which has gained widespread use in computer science in recent years, especially in the field of text mining and information retrieval. Since it was first proposed, it has received a large amount of attention and general interest among scientists in many research areas. It allows us to discover the mix of hidden or "latent" subjects that differs from one document to another in a given corpus. But since topic modeling usually requires the prior definition of some parameters - above all the number of topics k to be discovered -, model evaluation is decisive to identify an "optimal" set of parameters for the specific data. Latent Dirichlet allocation (LDA) and Bidirectional Encoder Representations from Transformers Topic (BerTopic) are the two most popular topic modeling techniques. LDA uses a probabilistic approach whereas BerTopic uses transformers (BERT embeddings) and class-based TF-IDF to create dense clusters.

Broadband Acoustic Metamaterial Design via Machine Learning

Acoustic metamaterials are engineered microstructures with special mechanical and acoustic properties enabling exotic effects such as wave steering, focusing and cloaking. In this research, we develop a new machine learning framework for predicting optimal metastructures such as planar configurations of scatterers with specific functionalities. Specifically, we implement this framework by combining probabilistic generative modeling with deep learning and propose two models: a conditional variational autoencoder (CVAE) and a supervised variational autoencoder (SVAE) model. As an application of the method, here we design an acoustic cloak considering a minimization of total scattering cross-section (TSCS) for a set of cylindrical obstacles. We work with the sets of cylindrical objects confined in a region of space and streamline the design of configurations with minimal TSCS, demonstrating broadband cloaking effect at discrete sets of wavenumbers. After establishing the artificial neural networks that are capable of learning the TSCS based on the location of cylinders, we discuss our inverse design algorithms, combining variational autoencoders and the Gaussian process, for predicting optimal arrangements of scatterers given the TSCS. We show results for up to eight cylinders and discuss the efficiency and other advantages of the machine learning approach.

Brain Decoding of Multiple Subjects for Estimating Visual Information Based on a Probabilistic Generative Model.

Brain decoding is a process of decoding human cognitive contents from brain activities. However, improving the accuracy of brain decoding remains difficult due to the unique characteristics of the brain, such as the small sample size and high dimensionality of brain activities. Therefore, this paper proposes a method that effectively uses multi-subject brain activities to improve brain decoding accuracy. Specifically, we distinguish between the shared information common to multi-subject brain activities and the individual information based on each subject’s brain activities, and both types of information are used to decode human visual cognition. Both types of information are extracted as features belonging to a latent space using a probabilistic generative model. In the experiment, an publicly available dataset and five subjects were used, and the estimation accuracy was validated on the basis of a confidence score ranging from 0 to 1, and a large value indicates superiority. The proposed method achieved a confidence score of 0.867 for the best subject and an average of 0.813 for the five subjects, which was the best compared to other methods. The experimental results show that the proposed method can accurately decode visual cognition compared with other existing methods in which the shared information is not distinguished from the individual information.

Attention-based generative adversarial network in medical imaging: A narrative review

As a popular probabilistic generative model, generative adversarial network (GAN) has been successfully used not only in natural image processing, but also in medical image analysis and computer-aided diagnosis. Despite the various advantages, the applications of GAN in medical image analysis face new challenges. The introduction of attention mechanisms, which resemble the human visual system that focuses on the task-related local image area for certain information extraction, has drawn increasing interest. Recently proposed transformer-based architectures that leverage self-attention mechanism encode long-range dependencies and learn representations that are highly expressive. This motivates us to summarize the applications of using transformer-based GAN for medical image analysis. We reviewed recent advances in techniques combining various attention modules with different adversarial training schemes, and their applications in medical segmentation, synthesis and detection. Several recent studies have shown that attention modules can be effectively incorporated into a GAN model in detecting lesion areas and extracting diagnosis-related feature information precisely, thus providing a useful tool for medical image processing and diagnosis. This review indicates that research on the medical imaging analysis of GAN and attention mechanisms is still at an early stage despite the great potential. We highlight the attention-based generative adversarial network is an efficient and promising computational model advancing future research and applications in medical image analysis.

Towards predicting liquid fuel physicochemical properties using molecular dynamics guided machine learning models

Accurate determination of fuel properties of complex mixtures over a wide range of pressure and temperature conditions is essential to utilizing alternative fuels. The present work aims to construct cheap-to-compute machine learning (ML) models to act as closure equations for predicting the physical properties of alternative fuels. Those models can be trained using the database from MD simulations and/or experimental measurements in a data-fusion-fidelity approach. Here, Gaussian Process (GP) and probabilistic generative models are adopted. GP is a popular non-parametric Bayesian approach to build surrogate models mainly due to its capacity to handle the aleatory and epistemic uncertainties. Generative models have shown the ability of deep neural networks employed with the same intent. In this work, ML analysis is focused on two particular properties, the fuel density and diffusion, but it can also be extended to other physicochemical properties. This study explores the versatility of the ML models to handle multi-fidelity data. The results show that ML models can predict accurately the fuel properties of a wide range of pressure and temperature conditions.

Community detection and reciprocity in networks by jointly modelling pairs of edges

Abstract To unravel the driving patterns of networks, the most popular models rely on community detection algorithms. However, these approaches are generally unable to reproduce the structural features of the network. Therefore, attempts are always made to develop models that incorporate these network properties beside the community structure. In this article, we present a probabilistic generative model and an efficient algorithm to both perform community detection and capture reciprocity in networks. Our approach jointly models pairs of edges with exact two-edge joint distributions. In addition, it provides closed-form analytical expressions for both marginal and conditional distributions. We validate our model on synthetic data in recovering communities, edge prediction tasks and generating synthetic networks that replicate the reciprocity values observed in real networks. We also highlight these findings on two real datasets that are relevant for social scientists and behavioural ecologists. Our method overcomes the limitations of both standard algorithms and recent models that incorporate reciprocity through a pseudo-likelihood approximation. The inference of the model parameters is implemented by the efficient and scalable expectation–maximization algorithm, as it exploits the sparsity of the dataset. We provide an open-source implementation of the code online.

Stochastic graph recurrent neural network

Representation learning over dynamic graphs has attracted much attention because of its wide applications. Recently, sequential probabilistic generative models have achieved impressive results because they can model data distributions. However, modeling the distribution of dynamic graphs is still extremely challenging. Existing methods usually ignore the mutual interference of stochastic states and deterministic states. Besides, the assumption that latent variables follow Gaussian distributions is usually inappropriate. To address these problems, we propose stochastic graph recurrent neural network (SGRNN), a sequential generative model for the representation learning over dynamic graphs. It separates stochastic states and deterministic states in the iterative process. To improve the flexibility of latent variables, we set the prior distribution and posterior distribution as semi-implicit distributions and propose DSI-SGRNN. In addition, to alleviate the KL-vanishing problem in SGRNN, a simple and interpretable structure is proposed based on the lower bound of KL-divergence. The proposed structure introduces a few extra parameters and can be implemented with a few lines of code modification. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed model.

Leveraging a Probabilistic PCA Model to Understand the Multivariate Statistical Network Monitoring Framework for Network Security Anomaly Detection

Network anomaly detection is a very relevant research area nowadays, especially due to its multiple applications in the field of network security. The boost of new models based on variational autoencoders and generative adversarial networks has motivated a reevaluation of traditional techniques for anomaly detection. It is, however, essential to be able to understand these new models from the perspective of the experience attained from years of evaluating network security data for anomaly detection. In this paper, we revisit anomaly detection techniques based on PCA from a probabilistic generative model point of view, and contribute a mathematical model that relates them. Specifically, we start with the probabilistic PCA model and explain its connection to the Multivariate Statistical Network Monitoring (MSNM) framework. MSNM was recently successfully proposed as a means of incorporating industrial process anomaly detection experience into the field of networking. We have evaluated the mathematical model using two different datasets. The first, a synthetic dataset created to better understand the analysis proposed, and the second, UGR'16, is a specifically designed real-traffic dataset for network security anomaly detection. We have drawn conclusions that we consider to be useful when applying generative models to network security detection.

Incompatibility of effects in general probabilistic models* *Paul Busch was an original coauthor of this paper, with Beneduci. Later, Neil Stevens also joined as a coauthor. This, then unpublished note, has since been referred to in, e.g., [, ]. The work of Stevens has since been removed, and Loveridge has made some additions and modifications. Paul passed away in 2018,

We give a necessary and sufficient condition for the incompatibility of a pair of effects in a general probabilistic model in which the state space is a total convex space, which can be obtained by minimising a real parameter. This has an interpretation in terms of the least noise that must be included to make the given pair compatible.

Deep learning formulation of ECGI evaluated on clinical data

Abstract Funding Acknowledgements Type of funding sources: Public grant(s) – EU funding. Main funding source(s): European funding from the ERC starting grant ECSTATIC (715093) and French funding from the National Research Agency grant IHU LIRYC (ANR-10-IAHU-04). Background Electrocardiographic Imaging (ECGI) is an exceptional resource in cardiology practice and research, allowing for non-invasive assessment of local cardiac electrical activities, through the acquisition of ECGs signals acquired with multi-electrodes vests. This approach is largely based on solving an ill-posed inverse problem. However, to date, there is no method sufficiently convincing to solve the inverse problem, to establish ECGI as the clinical modality of choice. Previously, we proposed a deep learning (DL) based method for ECGI reconstruction by exploiting multimodal information and prior knowledge from previous cases. Tested with synthetic data, the method proved to be effective and convincing, but clinical validation is lacking. Purpose To propose a model allowing for fast generation of activation maps from combined Body Surface Potentials (BSP) and imaging data acquired in patients. Methods The entire ECGI problem was reformulated with a probabilistic generative model, the Conditional Variational AutoEncoder (CVAE). During training, the model learned the conditional probability of the activation map based on conductivity maps and BSP of synthetic data. The model was constructed with convolutional layers, which allowed to extract the spatio-temporal correlations from the multimodal conditional data. To construct training and evaluation datasets, we proposed the following data preparation pipeline. First, the clinical data from 7 patients was processed, then the geometrical modelling of the torso and the heart was done. Fast personalized cardiac simulation using an eikonal model was applied to build the training dataset. Then the ground truth (GT) data was transferred to create the evaluation dataset. In addition, BSPs were processed to fit the simulated data onto the GT. The model was trained on the simulated data. To evaluate our method, we compared the GT activation map, captured by the CARTO system, with the reconstructed output of the trained model from the registered BSPs. Results Our model was trained on 5 different cardiac anatomies, with 4500 generated activation maps and BSPs. It was evaluated on two additional patients. The method allowed for the generation of activation maps from BSP signals in less than 1s. Comparison of our generated activation maps with the GT showed a good identification of the cardiac activation pattern (mean distance between generated and GT 32+/-8.3 ms). Validation from 7 activation onsets indicated a quite accurate onset localization (11.5+/- 8.6 mm). Conclusion Our DL ECGI solver allows for fast generation of activation maps from clinically-acquired BSPs. By probabilistically learning the spatio-temporal correlations and integrating anatomical information, the method goes beyond the current state-of-the-art, with comparison to invasive contact mapping data demonstrating accurate identification of activation patterns and a precise localization of the onset of focal activations.