Bayesian Graphical Models Research Articles

A Bayesian Graphical Model to Discover Latent Events from Twitter

Online social networks like Twitter and Facebook produce an overwhelming amount of information every day. However, research suggests that much of this content focuses on a reasonably sized set of ongoing events or topics that are both temporally and geographically situated. These patterns are especially observable when the data that is generated contains geospatial information, usually generated by a location enabled device such as a smartphone. In this paper, we consider a data set of 1.4 million geo-tagged tweets from a country during a large social movement, where social events and demonstrations occurred frequently. We use a probabilistic graphical model to discover these events within the data in a way that informs us of their spatial, temporal and topical focus. Quantitative analysis suggests that the streaming algorithm proposed in the paper uncovers both well-known events and lesser-known but important events that occurred within the timeframe of the dataset. In addition, the model can be used to predict the location and time of texts that do not have these pieces of information, which accounts for the much of the data on the web.

The Modified-Half-Normal distribution: Properties and an efficient sampling scheme

We introduce a new family of probability distributions that we call the Modified-Half-Normal distributions. It is supported on the positive part of the real line with its probability density proportional to the function We explore a number of its properties including showing the fact that the normalizing constant and moments of the distribution can be represented in terms of the Fox-Wright function. We demonstrate its relevance by showing its connection to a number of Bayesian statistical methods appearing from multiple areas of research such as Bayesian Binary regression, Analysis of Directional data, and Bayesian graphical model. The availability of its efficient sampling scheme is important to the success of such Bayesian procedures. Therefore, a major focus of this article is the development of methods for generating random samples from the Modified-Half-Normal distribution. To ensure efficiency, we prove that the constructed accept reject algorithms are “uniformly efficient” with high acceptance probability irrespective of the choice of the parameter specifications. Finally, we provide basic inference procedure for its parameters when analyzing data assuming the Modified-Half-Normal probability model.

Belief propagation for networks with loops.

Belief propagation is a widely used message passing method for the solution of probabilistic models on networks such as epidemic models, spin models, and Bayesian graphical models, but it suffers from the serious shortcoming that it works poorly in the common case of networks that contain short loops. Here, we provide a solution to this long-standing problem, deriving a belief propagation method that allows for fast calculation of probability distributions in systems with short loops, potentially with high density, as well as giving expressions for the entropy and partition function, which are notoriously difficult quantities to compute. Using the Ising model as an example, we show that our approach gives excellent results on both real and synthetic networks, improving substantially on standard message passing methods. We also discuss potential applications of our method to a variety of other problems.

Using Bayesian network technology to predict the semiconductor manufacturing yield rate in IoT

In the era of the knowledge economy, one of the key issues is how to integrate and use intelligent information systems to collect data and make valuable predictions to support business decisions. Intelligent information systems use artificial intelligence to enhance system performance, giving the enterprise a competitive advantage. This paper uses ontology technology for user requirements analysis—defining the class, slot, and instance of the ontology—and then designs the system architecture based on that ontology. In accordance with the firm’s requirements, we build a data warehouse system that is integrated with different data sources within the enterprise and that supports a web-based interface in the Internet of Things (IoT). The system also supports the standard queries, reports, summary tables, and datasets required for data mining. The proposed data mining method for manufacturing industries that use Bayesian network incorporates Bayesian theory and graphical models and can predict causal and probabilistic relationships among a set of variables. Our results bring information system functionality closer to satisfying the real-world needs of business. The proposed system can reduce production cycle times, increase the speed and accuracy with which production information is analyzed, and offer predictions that can be used for better business decisions. Data mining technology can improve the efficiency of manufacturing processes by using feedback data to tune the manufacturing parameters and improve the accuracy of yield rate predictions, giving the firm a greater competitive advantage.

A network psychometric approach to neurocognition in early Alzheimer's disease

In a typical pattern of Alzheimer's disease onset, episodic memory decline is predominant while decline in other neurocognitive domains is subsidiary or absent. Such descriptions refer to relationships between neurocognitive domains as well as deficits within domains. However, the former relationships are rarely statistically modelled. This study used psychometric network analysis to model relationships between neurocognitive variables in cognitive normality (CN), amnestic mild cognitive impairment (aMCI), and early Alzheimer's disease (eAD). Gaussian graphical models with extended Bayesian information criterion graphical lasso model selection and regularisation were used to estimate network models of neurocognitive and demographic variables in CN (n = 229), aMCI (n = 395), and eAD (n = 191) groups. The edge density, network strength and structure, centrality, and individual links of the network models were explored. Results indicated that while global strength did not differ, network structures differed across CN and eAD and aMCI and eAD groups, suggesting neurocognitive reorganisation across the eAD continuum. Episodic memory variables were most central (i.e., influential) in the aMCI network model, whereas processing speed and fluency variables were most central in the eAD network model. Additionally, putative clusters of memory, language and semantic variables, and attention, processing speed and working memory variables arose in the models for the clinical groups. This exploratory study shows how psychometric network analysis can be used to model the relationships between neurocognitive variables across the eAD continuum and to generate hypotheses for future (dis)confirmatory research.

The Network Constellation of Personality and Substance Use: Evolution from Early to Late Adolescence

There is a well–established link between substance use and four personality traits of anxiety–sensitivity, hopelessness, impulsivity, and sensation–seeking. However, construct–level models of personality may conceal indicator–level personality–outcome associations. The current study aims to investigate evolution of the network constellation of personality and cannabis/alcohol use from early to late adolescence. Data comes from the longitudinal Co–Venture cohort (N = 3800). Personality indicators, measured by Substance Use Risk Profile Scale (SURPS) items, and the frequency of cannabis/alcohol use were assessed at four consecutive years (13–17 years old). Network constellations of the SURPS items and cannabis/alcohol use were estimated using Bayesian Gaussian graphical models at four time points. Results highlighted the age–specific associations between personality indicators and substance use. The positive role of the sensation–seeking trait (e.g. attitude towards transgression) was constant, whereas the positive role of hopelessness indicators (e.g. not being enthusiastic about future) and the negative role of anxiety–sensitivity indicators (e.g. fear of having unusual body sensations) were more prominent at early adolescence. The current study provides a novel perspective on the network structure of personality and substance use in adolescence and suggests substance–specific and age–adjusted targets in intervention efforts. © 2020 European Association of Personality Psychology

Learning meta-knowledge for few-shot image emotion recognition

Previous studies have demonstrated that images are of great importance in attracting people’s attention and motivating them to take action. Various attributes (e.g., colors, aesthetics, and embedded objects) related to images are considered driving factors. Among which emotions in images, in particular, play a critical role in stimulating individuals, based on the Stimulus–Organism–Response theory. Consequently, many researchers put great efforts to understand image emotions, ranging from developing theoretical models to a broad spectrum of applications. Due to the complex and unstructured characteristics of images, identifying image emotions is challenging. Although some significant progress in image emotion classification has been achieved, inherent constraints still remain unaddressed. For example, acquiring a sufficiently large amount of labeled data to train a good model is costly and inevitably requires lots of human efforts. Besides, building a generalized model applicable to different datasets still needs a deep exploration. Image emotions are very subjective, which also makes such a classification task difficult. This paper proposes a general meta-learning framework for the few-shot image emotion classification, called Meta-IEC. Meta-IEC provides the capability of: (i) adapting to a similar dataset but new classes that have not been encountered before, and (ii) generalizing to a completely different dataset where emotion classes are unseen in the training dataset and only very few labeled images are available. Meta-IEC is also able to capture the uncertainty and ambiguity during the meta-testing, where we implement a hierarchical Bayesian graphical model to understand latent relationships among various parameters between meta-training and meta-testing. Extensive experiments on three commonly used datasets empirically demonstrate the superiority of our method over several state-of-the-art baselines. For example, our meta-learning based model can achieve performance improvement up to 5+%. We also provide some managerial implications on parameter sensitivity and label selection of meta-training and meta-testing.

A three-dimensional pattern recognition localization system based on a Bayesian graphical model

Access points in wireless local area networks are deployed in many indoor environments. Device-free wireless localization systems based on available received signal strength indicators have gained considerable attention recently because they can localize the people using commercial off-the-shelf equipment. Majority of localization algorithms consider two-dimensional models that cause low positioning accuracy. Although three-dimensional localization models are available, they possess high computational and localization errors, given their use of numerous reference points. In this work, we propose a three-dimensional indoor localization system based on a Bayesian graphical model. The proposed model has been tested through experiments based on fingerprinting technique which collects received signal strength indicators from each access point in an offline training phase and then estimates the user location in an online localization phase. Results indicate that the proposed model achieves a high localization accuracy of more than 25% using reference points fewer than that of benchmarked algorithms.

BGGM: Bayesian Gaussian Graphical Models in R

BGGM: Bayesian Gaussian Graphical Models in R

Scalable Bayesian matrix normal graphical models for brain functional networks.

Recently, there has been an explosive growth in graphical modeling approaches for estimating brain functional networks. In a detailed study, we show that surprisingly, standard graphical modeling approaches for fMRI data may not yield accurate estimates of the brain network due to the inability to suitably account for temporal correlations. We propose a novel Bayesian matrix normal graphical model that jointly models the temporal covariance and the brain network under a separable structure for the covariance to obtain improved estimates. The approach is implemented via an efficient optimization algorithm that computes the maximum-a-posteriori network estimates having desirable theoretical properties and which is scalable to high dimensions. The proposed method leads to substantial gains in network estimation accuracy compared to standard brain network modeling approaches as illustrated via extensive simulations. We apply the method to resting state fMRI data from the Human Connectome Project involving a large number of time scans and brain regions, to study the relationships between fluid intelligence and functional connectivity, where it is not computationally feasible to apply existing matrix normal graphical models. Our proposed approach led to the detection of differences in connectivity between high and low fluid intelligence groups, whereas these differences were less pronounced or absent using the graphicallasso.

Community Topic Usage in Online Social Media

Humans have a natural tendency to form social groups, and individual behaviours are thought to be strongly influenced by a salient sense of belonging to one or more such groups. It can be expected, therefore, that there will be behaviours that are specific to the group(s) to which a person currently feels they are interacting with and that some of these behaviours will manifest in topics and patterns of linguistic style associated with those groups. Here we explore this idea by attempting to identify group specific patterns of language usage in social media data from Twitter and Reddit. Topic models are used to infer patterns of language usage and group structures are either provided with the data (Reddit) inferred from the follower network (Twitter). We apply a Bayesian graphical model to infer community-topic associations, finding that substantially more coherent associations can often be identified than with a naive probability-based approach. Strong associations are found between groups and topics with both approaches, indicating that the methods used to (independently) identify groups and topics represent real underlying patterns of social communication and promising fruitful investigation of human social behaviour using these or similar techniques.

Bayesian graphical model based optimal decision-making for fault diagnosis of critical induction motors in industrial applications

Bayesian graphical model based optimal decision-making for fault diagnosis of critical induction motors in industrial applications

A novel CMB component separation method: hierarchical generalized morphological component analysis

ABSTRACT We present a novel technique for cosmic microwave background (CMB) foreground subtraction based on the framework of blind source separation. Inspired by previous work incorporating local variation to generalized morphological component analysis (GMCA), we introduce hierarchical GMCA (HGMCA), a Bayesian hierarchical graphical model for source separation. We test our method on Nside = 256 simulated sky maps that include dust, synchrotron, free–free, and anomalous microwave emission, and show that HGMCA reduces foreground contamination by $25{{\ \rm per\ cent}}$ over GMCA in both the regions included and excluded by the Planck UT78 mask, decreases the error in the measurement of the CMB temperature power spectrum to the 0.02–0.03 per cent level at ℓ > 200 (and $\lt 0.26{{\ \rm per\ cent}}$ for all ℓ), and reduces correlation to all the foregrounds. We find equivalent or improved performance when compared to state-of-the-art internal linear combination type algorithms on these simulations, suggesting that HGMCA may be a competitive alternative to foreground separation techniques previously applied to observed CMB data. Additionally, we show that our performance does not suffer when we perturb model parameters or alter the CMB realization, which suggests that our algorithm generalizes well beyond our simplified simulations. Our results open a new avenue for constructing CMB maps through Bayesian hierarchical analysis.

Multilayer network analysis of oil linkages

Summary This manuscript proposes a new approach for unveiling existing linkages within the international oil market across multiple driving factors beyond production. A multilayer, multicountry network is extracted through a novel Bayesian graphical vector autoregressive model, which allows for a more comprehensive, dynamic representation of the network linkages than do traditional or static pairwise Granger-causal inference approaches. Building on previous work, the layers of the network include country- and region-specific oil production levels and rigs, both through simultaneous and lagged temporal dependences among key factors, while controlling for oil prices and a world economic activity index. The proposed approach extracts relationships across all variables through a dynamic, cross-regional network. This approach is highly scalable and adjusts for time-evolving linkages. The model outcome is a set of time-varying graphical networks that unveil both static representations of world oil linkages and variations in microeconomic relationships both within and between oil producers. An example is provided, illustrating the evolution of intra- and inter-regional relationships for two major interconnected oil producers: the United States, with a regional decomposition of its production and rig deployment, and the Arabian Peninsula and key Middle Eastern producers, with a country-based decomposition of production and rig deployment, while controlling for oil prices and global economic indices. Production is less affected by concurrent changes in oil prices and the overall economy than rigs. However, production is a lagged driver for prices, rather than rigs, which indicates that the linkage between rigs and production may not be fully accounted for in the markets.

Structural learning of contemporaneous dependencies in graphical VAR models

An objective Bayes approach based on graphical modeling is proposed to learn the contemporaneous dependencies among multiple time series within the framework of Vector Autoregressive (VAR) models. Assuming that, at any time, the covariance matrix is Markov with respect to the same decomposable graph, it is shown that the likelihood of a graphical VAR can be factorized as an ordinary (decomposable) graphical model. Additionally, using a fractional Bayes factor approach, the marginal likelihood is obtained in closed form, and an MCMC algorithm for Bayesian graphical model determination with limited computational burden is presented. The method is validated through a simulation study and applied to a real data set concerning active users of the Earthquake Network application for smartphones.

Enhancing Personalized Recommendation by Implicit Preference Communities Modeling

Recommender systems aim to capture user preferences and provide accurate recommendations to users accordingly. For each user, there usually exist others with similar preferences, and a collection of users may also have similar preferences with each other, thus forming a community. However, such communities may not necessarily be explicitly given, and the users inside the same communities may not know each other; they are formally defined and named Implicit Preference Communities (IPCs) in this article. By enriching user preferences with the information of other users in the communities, the performance of recommender systems can also be enhanced. Historical explicit ratings are a good resource to construct the IPCs of users but is usually sparse. Meanwhile, user preferences are easily affected by their social connections, which can be jointly used for IPC modeling with the ratings. However, this imposes two challenges for model design. First, the rating and social domains are heterogeneous; thus, it is challenging to coordinate social information and rating behaviors for a same learning task. Therefore, transfer learning is a good strategy for IPC modeling. Second, the communities are not explicitly labeled, and existing supervised learning approaches do not fit the requirement of IPC modeling. As co-clustering is an effective unsupervised learning approach for discovering block structures in high-dimensional data, it is a cornerstone for discovering the structure of IPCs. In this article, we propose a recommendation model with Implicit Preference Communities from user ratings and social connections. To tackle the unsupervised learning limitation, we design a Bayesian probabilistic graphical model to capture the IPC structure for recommendation. Meanwhile, following the spirit of transfer learning, both rating behaviors and social connections are introduced into the model by parameter sharing. Moreover, Gibbs sampling-based algorithms are proposed for parameter inferences of the models. Furthermore, to meet the need for online scenarios when the data arrive sequentially as a stream, a novel online sampling-based parameter inference algorithm for recommendation is proposed. To the best of our knowledge, this is the first attempt to propose and formally define the concept of IPC.

Adaptive Local Low-rank Matrix Approximation for Recommendation

Low-rank matrix approximation (LRMA) has attracted more and more attention in the community of recommendation. Even though LRMA-based recommendation methods (including Global LRMA and Local LRMA) obtain promising results, they suffer from the complicated structure of the large-scale and sparse rating matrix, especially when the underlying system includes a large set of items with various types and a huge amount of users with diverse interests. Thus, they have to predefine the important parameters, such as the rank of the rating matrix and the number of submatrices. Moreover, most existing Local LRMA methods are usually designed in a two-phase separated framework and do not consider the missing mechanisms of rating matrix. In this article, a non-parametric unified Bayesian graphical model is proposed for A daptive Lo cal low-rank M atrix A pproximation ( ALoMA ). ALoMA has ability to simultaneously identify rating submatrices, determine the optimal rank for each submatrix, and learn the submatrix-specific user/item latent factors. Meanwhile, the missing mechanism is adopted to characterize the whole rating matrix. These four parts are seamlessly integrated and enhance each other in a unified framework. Specifically, the user-item rating matrix is adaptively divided into proper number of submatrices in ALoMA by exploiting the Chinese Restaurant Process. For each submatrix, by considering both global/local structure information and missing mechanisms, the latent user/item factors are identified in an optimal latent space by adopting automatic relevance determination technique. We theoretically analyze the model’s generalization error bounds and give an approximation guarantee. Furthermore, an efficient Gibbs sampling-based algorithm is designed to infer the proposed model. A series of experiments have been conducted on six real-world datasets ( Epinions , Douban , Dianping , Yelp , Movielens (10M), and Netflix ). The results demonstrate that ALoMA outperforms the state-of-the-art LRMA-based methods and can easily provide interpretable recommendation results.

The role of uncertainty measures on the returns of gold

By utilizing Bayesian Graphical Structural Vector Autoregression model, we show that changes in geopolitical risks and the U.S. real effective exchange rate significantly affect Gold returns. These results are consistent across different frequency bands in short, medium, and long terms.

Truth finding by reliability estimation on inconsistent entities for heterogeneous data sets

An important task in big data integration is to derive accurate data records from noisy and conflicting values collected from multiple sources. Most existing truth finding methods assume that the reliability is consistent on the whole data set, ignoring the fact that different attributes, objects and object groups may have different reliabilities even wrt the same source. These reliability differences are caused by the hardness differences in obtaining attribute values, non-uniform updates to objects and the differences in group privileges. This paper addresses the problem how to compute truths by effectively estimating the reliabilities of attributes, objects and object groups in a multi-source heterogeneous data environment. We first propose an optimization framework TFAR, its implementation and Lagrangian duality solution for Truth Finding by Attribute Reliability estimation. We then present a Bayesian probabilistic graphical model TFOR and an inference algorithm applying Collapsed Gibbs Sampling for Truth Finding by Object Reliability estimation. Finally we give an optimization framework TFGR and its implementation for Truth Finding by Group Reliability estimation. All these models lead to a more accurate estimation of the respective attribute, object and object group reliabilities, which in turn can achieve a better accuracy in inferring the truths. Experimental results on both real data and synthetic data show that our methods have better performance than the state-of-art truth discovery methods.

Abstract 2441: Association between NRF1 regulatory gene networks and cancer and aging

Abstract There is a growing convergence between molecular aspects of aging and cancer. Cellular senescence acts as a barrier through preventing the proliferation of cells under oncogenic stress. Therefore, we decided to examine the role of nuclear respiratory factor 1 (NRF1) transcription factor in human brain cancer and brain aging. In contrast to higher expression of NRF1 in human glioma tumors, lower NRF1 levels are observed in Alzheimer’s disease (AD) brain tissue. Using the glioma and AD transcriptomic data of NRF1 target genes, we conducted Transcription Factor Target Enrichment Analysis that detects transcription factor (TF) activity based on quantification of the collective transcriptional activation of their target genes. NRF1 TF activity was significantly lower in AD brain tissues from four independent cohorts compared to non-AD brain tissues. In contrast, we observed higher NRF1 transcription factor activity in glioma. NRF1 activity as well as expression of several cell cycle pathway genes including CDK1,2,6,9 and NRF1 target genes such as NR2E1, APOE and MAPT are significantly altered in these two population cohorts. We also discovered that expression of NRF1, RB1 and E2F influenced survival in glioma. Specifically, the mean overall survival for patients with a high NRF1, E2F1,2,6,7,8 and RB1 gene signature was 20.91 months (95% CI = 9.37-32.46), compared to 101.99 months (95% CI = 74.28-129.70), in high NRF1 expressing patients that lacked the specific gene signature. NRF1 based gene signatures combined with IDH1 mutation status are necessary and sufficient for explaining the observed differences in survival curves. We examined the association of NRF1, the E2Fs and RB1 through Bayesian graphical modeling and observed gender differences in gene networks between NRF1 and E2F family members in the glioma patient sub-populations as compared to normal aging human brain samples. Furthermore, the differentially expressed NRF1 regulated genes are part of the KEGG pathways associated with spliceosome, ribosome and lysosome function as well as a diverse array of other functions are enriched, indicating ways in which NRF1 may be involved in glioma biology. In summary these findings support our postulate that NRF1 is an important transcription factor in the human brain, where it plays an active role in both aging and cancer. Citation Format: Kaumudi Bhawe, Quentin Felty, Changwon Yoo, Deodutta Roy. Association between NRF1 regulatory gene networks and cancer and aging [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 2441.