Undirected Probabilistic Graphical Models Research Articles

A Joint Community Detection Model: Integrating Directed and Undirected Probabilistic Graphical Models via Factor Graph With Attention Mechanism

Directed and undirected probabilistic graphical models have been successfully used in community detection in recent years, but existing graphical model based methods usually only use one type of probabilistic graphical model to discover communities. However, directed and undirected graphical models have their own advantages for characterizing different network information (attribute information and network topology). Intuitively, we can make use of the merit of both kinds of models by combining them into a unified model. However, combining directed and undirected graphical models is difficult, as they have different properties which prevent parameter sharing and joint training. In this article, we propose a unified model which integrates directed and undirected graphical models by transforming both them into factor graph. In addition, as network topology and attribute information may contain different degrees of noises, we add a selective attention layer to learn the reliable weight of each type of information source in node granularity. For training the model, we derive an iterative belief propagation algorithm to train all the parameters simultaneously. Extensive experiments on real networks and artificial benchmarks show the superiority of our approach over existing methods.

Abstract 12957: Determining Disease Co-Occurrence Architecture of Hypertensive Heart Disease in Penn Medicine Biobank Using Longitudinal EHR Data Linked With PMBB Participants

An estimated 1.2 billion individuals worldwide suffer from hypertension. Hypertensive heart disease (HHD) refers to structural alterations, such as left ventricular hypertrophy in the heart due to prolonged hypertension and can lead to ischemic heart disease, heart failure, and arrhythmias. HHD represents a transitory subclinical condition between the onset of hypertension and cardiovascular disease, typically remaining asymptomatic until the time of adverse cardiac event. The risk of HHD-related diseases is heightened due to hypertension-associated comorbidities such as dyslipidemia and renal failure. The complex etiology underlying HHD makes it challenging to predict outcomes in patients. The goal of this study is to leverage longitudinal electronic health record (EHR) data to identify the conditions and comorbidities that influence HHD outcomes for prognostication. We defined an HHD patient cohort (n = 4,268) in the Penn Medicine BioBank dataset using diagnosis code ICD-10: I11* for HHD. We constructed a disease-disease map of HHD comorbidities using the Ising model, a type of Markov Random Field undirected probabilistic graphical model. We estimated the co-occurrence of disease pairs by calculating pairwise similarities between diseases as relative risk and conducting network analyses. Our results reflect clusters of known disease-disease connections, including interactions between endocrine and nervous system diseases, as well as novel pairwise disease interactions between genitourinary and musculoskeletal diseases. Our findings shed light on the effect of comorbid conditions on the trajectory of HHD adverse outcomes. Using the probabilistic model of disease co-occurrence from these results, we can identify key comorbidities that predict adverse outcomes among HHD patients. This work lays the foundation to integrate rare and common genetic variants into the Ising model using a gene-based binning approach. Incorporating rare loss-of-function variants will enable us to evaluate shared genetic connections between HHD comorbidities. In conclusion, we used EHR data to identify the genetic and non-genetic connections between comorbidities of HHD to develop a predictive metric for the prevention and clinical monitoring of HHD.

Toward sampling from undirected probabilistic graphical models using a D-Wave quantum annealer

A D-Wave quantum annealer (QA) having a 2048 qubit lattice, with no missing qubits and couplings, allowed embedding of a complete graph of a restricted Boltzmann machine (RBM). A handwritten digit OptDigits dataset having 8 × 7 pixels of visible units was used to train the RBM using classical contrastive divergence. Embedding of the classically trained RBM into the D-Wave lattice was used to demonstrate that the QA offers a high-efficiency alternative to the classical Markov chain Monte Carlo (MCMC) for reconstructing missing labels of the test images as well as a generative model on a classically trained RBM. At any training iteration, the D-Wave-based classification had classification error less than half of that in MCMC. The main goal of this study was to investigate the quality of the QA sample from the RBM model probability distribution and compare it to a classical MCMC during the Gibbs sampling traditionally used in RBM training. For the OptDigits dataset, the states in the D-Wave sample belonged to about twice as many local valleys (LVs) as in the MCMC sample. All the lowest energy (the highest joint probability), local minima in the MCMC sample were also found by the D-Wave. The D-Wave missed many of the higher-energy LVs, while also finding many “new” LVs consistently missed by the MCMC. It was established that the “new” LVs that the D-Wave finds are important for the model distribution in terms of the energy of the corresponding local minima, the width of the LVs and the height of the escape barrier.

Automatic cytoplasm and nuclei segmentation for color cervical smear image using an efficient gap-search MRF

Accurate and effective cervical smear image segmentation is required for automated cervical cell analysis systems. Thus, we proposed a novel superpixel-based Markov random field (MRF) segmentation framework to acquire the nucleus, cytoplasm and image background of cell images. We seek to classify color non-overlapping superpixel-patches on one image for image segmentation. This model describes the whole image as an undirected probabilistic graphical model and was developed using an automatic label-map mechanism for determining nuclear, cytoplasmic and background regions. A gap-search algorithm was designed to enhance the model efficiency. Data show that the algorithms of our framework provide better accuracy for both real-world and the public Herlev datasets. Furthermore, the proposed gap-search algorithm of this model is much more faster than pixel-based and superpixel-based algorithms.

Probabilistic graphical models in modern social network analysis

The advent and availability of technology has brought us closer than ever through social networks. Consequently, there is a growing emphasis on mining social networks to extract information for knowledge and discovery. However, methods for social network analysis (SNA) have not kept pace with the data explosion. In this review, we describe directed and undirected probabilistic graphical models (PGMs), and highlight recent applications to social networks. PGMs represent a flexible class of models that can be adapted to address many of the current challenges in SNA. In this work, we motivate their use with simple and accessible examples to demonstrate the modeling and connect to theory. In addition, recent applications in modern SNA are highlighted, including the estimation and quantification of importance, propagation of influence, trust (and distrust), link and profile prediction, privacy protection, and news spread through microblogging. Applications are selected to demonstrate the flexibility and predictive capabilities of PGMs in SNA. Finally, we conclude with a discussion of challenges and opportunities for PGMs in social networks.

A Semiautomated Probabilistic Framework for Tree-Cover Delineation From 1-m NAIP Imagery Using a High-Performance Computing Architecture

Accurate tree-cover estimates are useful in deriving above-ground biomass density estimates from very high resolution (VHR) satellite imagery data. Numerous algorithms have been designed to perform tree-cover delineation in high-to-coarse-resolution satellite imagery, but most of them do not scale to terabytes of data, typical in these VHR data sets. In this paper, we present an automated probabilistic framework for the segmentation and classification of 1-m VHR data as obtained from the National Agriculture Imagery Program (NAIP) for deriving tree-cover estimates for the whole of Continental United States, using a high-performance computing architecture. The results from the classification and segmentation algorithms are then consolidated into a structured prediction framework using a discriminative undirected probabilistic graphical model based on conditional random field, which helps in capturing the higher order contextual dependence relations between neighboring pixels. Once the final probability maps are generated, the framework is updated and retrained by incorporating expert knowledge through the relabeling of misclassified image patches. This leads to a significant improvement in the true positive rates and reduction in false positive rates (FPRs). The tree-cover maps were generated for the state of California, which covers a total of 11 095 NAIP tiles and spans a total geographical area of 163 696 sq. miles. Our framework produced correct detection rates of around 88% for fragmented forests and 74% for urban tree-cover areas, with FPRs lower than 2% for both regions. Comparative studies with the National Land-Cover Data algorithm and the LiDAR high-resolution canopy height model showed the effectiveness of our algorithm for generating accurate high-resolution tree-cover maps.

Discovering Typed Communities in Mobile Social Networks

Mobile social networks, which consist of mobile users who communicate with each other using cell phones, are reflections of people’s interactions in social lives. Discovering typed communities (e.g., family communities or corporate communities) in mobile social networks is a very promising problem. For example, it can help mobile operators to determine the target users for precision marketing. In this paper we propose discovering typed communities in mobile social networks by utilizing the labels of relationships between users. We use the user logs stored by mobile operators, including communication and user movement records, to collectively label all the relationships in a network, by employing an undirected probabilistic graphical model, i.e., conditional random fields. Then we use two methods to discover typed communities based on the results of relationship labeling: one is simply retaining or cutting relationships according to their labels, and the other is using sophisticated weighted community detection algorithms. The experimental results show that our proposed framework performs well in terms of the accuracy of typed community detection in mobile social networks.

Probabilistic Image Modeling With an Extended Chain Graph for Human Activity Recognition and Image Segmentation

Chain graph (CG) is a hybrid probabilistic graphical model (PGM) capable of modeling heterogeneous relationships among random variables. So far, however, its application in image and video analysis is very limited due to lack of principled learning and inference methods for a CG of general topology. To overcome this limitation, we introduce methods to extend the conventional chain-like CG model to CG model with more general topology and the associated methods for learning and inference in such a general CG model. Specifically, we propose techniques to systematically construct a generally structured CG, to parameterize this model, to derive its joint probability distribution, to perform joint parameter learning, and to perform probabilistic inference in this model. To demonstrate the utility of such an extended CG, we apply it to two challenging image and video analysis problems: human activity recognition and image segmentation. The experimental results show improved performance of the extended CG model over the conventional directed or undirected PGMs. This study demonstrates the promise of the extended CG for effective modeling and inference of complex real-world problems.

Gaussian graphical modeling reconstructs pathway reactions from high-throughput metabolomics data

BackgroundWith the advent of high-throughput targeted metabolic profiling techniques, the question of how to interpret and analyze the resulting vast amount of data becomes more and more important. In this work we address the reconstruction of metabolic reactions from cross-sectional metabolomics data, that is without the requirement for time-resolved measurements or specific system perturbations. Previous studies in this area mainly focused on Pearson correlation coefficients, which however are generally incapable of distinguishing between direct and indirect metabolic interactions.ResultsIn our new approach we propose the application of a Gaussian graphical model (GGM), an undirected probabilistic graphical model estimating the conditional dependence between variables. GGMs are based on partial correlation coefficients, that is pairwise Pearson correlation coefficients conditioned against the correlation with all other metabolites. We first demonstrate the general validity of the method and its advantages over regular correlation networks with computer-simulated reaction systems. Then we estimate a GGM on data from a large human population cohort, covering 1020 fasting blood serum samples with 151 quantified metabolites. The GGM is much sparser than the correlation network, shows a modular structure with respect to metabolite classes, and is stable to the choice of samples in the data set. On the example of human fatty acid metabolism, we demonstrate for the first time that high partial correlation coefficients generally correspond to known metabolic reactions. This feature is evaluated both manually by investigating specific pairs of high-scoring metabolites, and then systematically on a literature-curated model of fatty acid synthesis and degradation. Our method detects many known reactions along with possibly novel pathway interactions, representing candidates for further experimental examination.ConclusionsIn summary, we demonstrate strong signatures of intracellular pathways in blood serum data, and provide a valuable tool for the unbiased reconstruction of metabolic reactions from large-scale metabolomics data sets.

Learning generative models for protein fold families

We introduce a new approach to learning statistical models from multiple sequence alignments (MSA) of proteins. Our method, called GREMLIN (Generative REgularized ModeLs of proteINs), learns an undirected probabilistic graphical model of the amino acid composition within the MSA. The resulting model encodes both the position-specific conservation statistics and the correlated mutation statistics between sequential and long-range pairs of residues. Existing techniques for learning graphical models from MSA either make strong, and often inappropriate assumptions about the conditional independencies within the MSA (e.g., Hidden Markov Models), or else use suboptimal algorithms to learn the parameters of the model. In contrast, GREMLIN makes no a priori assumptions about the conditional independencies within the MSA. We formulate and solve a convex optimization problem, thus guaranteeing that we find a globally optimal model at convergence. The resulting model is also generative, allowing for the design of new protein sequences that have the same statistical properties as those in the MSA. We perform a detailed analysis of covariation statistics on the extensively studied WW and PDZ domains and show that our method out-performs an existing algorithm for learning undirected probabilistic graphical models from MSA. We then apply our approach to 71 additional families from the PFAM database and demonstrate that the resulting models significantly out-perform Hidden Markov Models in terms of predictive accuracy.

Protein Design by Sampling an Undirected Graphical Model of Residue Constraints

This paper develops an approach for designing protein variants by sampling sequences that satisfy residue constraints encoded in an undirected probabilistic graphical model. Due to evolutionary pressures on proteins to maintain structure and function, the sequence record of a protein family contains valuable information regarding position-specific residue conservation and coupling (or covariation) constraints. Representing these constraints with a graphical model provides two key benefits for protein design: a probabilistic semantics enabling evaluation of possible sequences for consistency with the constraints, and an explicit factorization of residue dependence and independence supporting efficient exploration of the constrained sequence space. We leverage these benefits in developing two complementary MCMC algorithms for protein design: constrained shuffling mixes wild-type sequences positionwise and evaluates graphical model likelihood, while component sampling directly generates sequences by sampling clique values and propagating to other cliques. We apply our methods to design WW domains. We demonstrate that likelihood under a model of wild-type WWs is highly predictive of foldedness of new WWs. We then show both theoretical and rapid empirical convergence of our algorithms in generating high-likelihood, diverse new sequences. We further show that these sequences capture the original sequence constraints, yielding a model as predictive of foldedness as the original one.