Bernoulli Mixture Model Research Articles

Identification of genetic subtypes in follicular lymphoma

Follicular lymphoma (FL) exhibits considerable variability in biological features and clinical trajectories across patients. To dissect the diversity of FL, we utilized a Bernoulli mixture model to identify genetic subtypes in 713 pre-treatment tumor tissue samples. Our analysis revealed the existence of five subtypes with unique genetic profiles that correlated with clinicopathological characteristics. The clusters were enriched in specific mutations as follows: CS (CREBBP and STAT6), TT (TNFAIP3 and TP53), GM (GNA13 and MEF2B), Q (quiescent, for low mutation burden), and AR (mutations of mTOR pathway-related genes). The subtype Q was enriched for patients with stage I disease and associated with a lower proliferative history than the other subtypes. The AR subtype was unique in its enrichment for IgM-expressing FL cases and was associated with advanced-stage and more than 4 nodal sites. The existence of subtypes was validated in an independent cohort of 418 samples from the GALLIUM trial. Notably, patients assigned to the TT subtype consistently experienced inferior progression-free survival when treated with immunochemotherapy. Our findings offer insight into core pathways distinctly linked with each FL cluster and are expected to be informative in the era of targeted therapies.

Association between comorbidity clusters and mortality in patients with cancer: A machine learning analysis of data from the US National Health and Nutrition Examination Survey 1999–2018.

11132 Background: Multimorbidity is common among patients with cancer; however, research on cancer prognosis has predominantly focused on cancers in isolation. This study identified comorbidity clusters among patients with cancer using machine learning and examine their associations with survival outcomes in a nationally representative sample of the US. Methods: This study used 10 survey cycles of the National Health and Nutrition Examination Survey from 1999 to 2018. Participants aged ≥20 years with a self-reported history of cancer were included. Comorbidities were ascertained through self-reports and quantitative measurements. Machine learning techniques, including Bernoulli mixture model and partition-based methods, were used to identify comorbidity clusters. Cox proportional hazards models were used to analyze the associations between comorbidity clusters and mortality outcomes, including all-cause mortality and cause-specific mortality (cancer, cardiovascular disease [CVD], and respiratory diseases), adjusting for relevant covariates. Results: The study included 4,390 participants. Four comorbidity clusters were identified: Low Comorbidity, Metabolic, CVD, and Respiratory. After adjusting for confounders, participants in the Respiratory Cluster had the highest risk of all-cause mortality (adjusted hazard ratio[aHR]=1.62, 95% confidence interval [CI]=1.26-2.08, p<0.001), followed by the CVD Cluster (aHR=1.50, 95%CI = 1.26-1.80, p<0.001) and the Metabolic Cluster (aHR=1.15, 95%CI=1.02-1.29, p=0.03) compared to the Low Comorbidity Cluster. The Metabolic, CVD, and Respiratory clusters were associated with higher risks of cardiovascular disease-related mortality (aHR=1.48-3.05, p<0.003), but no significant differences in cancer mortality were observed among the clusters. The effects of comorbidity clusters on all-cause mortality were modified by income-to-poverty ratio ( p for interaction=0.04), diet quality ( p for interaction=0.02), time since cancer diagnosis ( p for interaction=0.009), and cancer prognosis ( p for interaction=0.005). Conclusions: High comorbidity burden clusters showed increased all-cause and CVD-related mortality. Moreover, diet quality and socioeconomic status modified these associations. Machine learning approaches can provide valuable insights into complex multimorbidity profiles in patients with cancer. Further research is needed to deepen understanding of the relationships between multimorbidity, mortality, and cancer-specific outcomes.

Network Inference Using the Hub Model and Variants

Statistical network analysis primarily focuses on inferring the parameters of an observed network. In many applications, especially in the social sciences, the observed data is the groups formed by individual subjects. In these applications, the network is itself a parameter of a statistical model. Zhao and Weko propose a model-based approach, called the hub model, to infer implicit networks from grouping behavior. The hub model assumes that each member of the group is brought together by a member of the group called the hub. The set of members which can serve as a hub is called the hub set. The hub model belongs to the family of Bernoulli mixture models. Identifiability of Bernoulli mixture model parameters is a notoriously difficult problem. This article proves identifiability of the hub model parameters and estimation consistency under mild conditions. Furthermore, this article generalizes the hub model by introducing a model component that allows hubless groups in which individual nodes spontaneously appear independent of any other individual. We refer to this additional component as the null component. The new model bridges the gap between the hub model and the degenerate case of the mixture model—the Bernoulli product. Identifiability and consistency are also proved for the new model. In addition, a penalized likelihood approach is proposed to estimate the hub set when it is unknown. Supplementary materials for this article are available online.

Stratified importance sampling for a Bernoulli mixture model of portfolio credit risk

Stratified importance sampling for a Bernoulli mixture model of portfolio credit risk

On the Hierarchical Bernoulli Mixture Model Using Bayesian Hamiltonian Monte Carlo

The model developed considers the uniqueness of a data-driven binary response (indicated by 0 and 1) identified as having a Bernoulli distribution with finite mixture components. In social science applications, Bernoulli’s constructs a hierarchical structure data. This study introduces the Hierarchical Bernoulli mixture model (Hibermimo), a new analytical model that combines the Bernoulli mixture with hierarchical structure data. The proposed approach uses a Hamiltonian Monte Carlo algorithm with a No-U-Turn Sampler (HMC/NUTS). The study has performed a compatible syntax program computation utilizing the HMC/NUTS to analyze the Bayesian Bernoulli mixture aggregate regression model (BBMARM) and Hibermimo. In the model estimation, Hibermimo yielded a result of ~90% compliance with the modeling of each district and a small Widely Applicable Information Criteria (WAIC) value.

Dependent Credit Default Risk Modelling Using Bernoulli Mixture Models

Generalized Linear Mixed Models (GLMMs) can be used to model the occurrence of defaults in a loan or bond portfolio. In this paper, we used a Bernoulli mixture model, a type of GLMMs, to model the dependency of default events. We discussed how Bernoulli mixture models can be used to model portfolio credit default risk, with the probit normal distribution as the link function. The general mathematical framework of the GLMMs was examined, with a particular focus on using Bernoulli mixture models to model credit default risk measures. We showed how GLMMs can be mapped into Bernoulli mixture models. An important aspect in portfolio credit default modelling is the dependence among default events, and in the GLMM setting, this may be captured using the so called random effects. Both fixed and random effects influence default probabilities of firms and these are taken as the systemic risk of the portfolio. After describing the model, we also conducted an empirical study for the applicability of our model using Standard and Poor’s data incorporating rating category (fixed effect) and time (random effect) as components of the model that constitute to the systemic risk of the portfolio. We were able to find the estimates of the model parameters using the Maximum Likelihood (ML) estimation method.

Multiple-instance Learning based on Bernoulli Mixture Model

Multiple-instance learning (MIL) is a form of weakly supervised learning. Its instances are arranged in groups (called bags), and labels are provided for the entire bag after training. The classic MIL method represents examples with pre-calculated features, and the classification process is also cumbersome. In this paper, we use neural networks to extract features, parameterize all transformations, and use Bernoulli mixture model to construct MIL models for baggage tags, using simpler network structures and more accurately solving these problems. Experiments show that our results can be competitive with the classical MIL algorithm on the MINST dataset.

Importance sampling with splitting for portfolio credit risk

We consider a credit portfolio with highly skewed exposures. In the portfolio, small number of obligors have very high exposures compared to the others. For the Bernoulli mixture model with highly skewed exposures, we propose a new importance sampling scheme to estimate the tail loss probability over a threshold and the corresponding expected shortfall. We stratify the sample space of the default events into two subsets. One consists of the events that the obligors with heavy exposures default simultaneously. We expect that typical tail loss events belong to the set. In our proposed scheme, the tail loss probability and the expected shortfall corresponding to this type of events are estimated by a conditional Monte Carlo, which results in variance reduction. We analyze the properties of the proposed scheme mathematically. In numerical study, the performance of the proposed scheme is compared with an existing importance sampling method.

Supervised mixture of experts models for population health.

We propose a machine learning driven approach to derive insights from observational healthcare data to improve public health outcomes. Our goal is to simultaneously identify patient subpopulations with differing health risks and to find those risk factors within each subpopulation. We develop two supervised mixture of experts models: a Supervised Gaussian Mixture model (SGMM) for general features and a Supervised Bernoulli Mixture model (SBMM) tailored to binary features. We demonstrate the two approaches on an analysis of high cost drivers of Medicaid expenditures for inpatient stays. We focus on the three diagnostic categories that accounted for the highest percentage of inpatient expenditures in New York State (NYS) in 2016. When compared with state-of-the-art learning methods (random forests, boosting, neural networks), our approaches provide comparable prediction performance while also extracting insightful subpopulation structure and risk factors. For problems with binary features the proposed SBMM provides as good or better performance than alternative methods while offering insightful explanations. Our results indicate the promise of such approaches for extracting population health insights from electronic health care records.

On the modeling of the new student acceptance status through science and technology written test using bernoulli mixture model

This research aimed to model StudentAcceptance Status at the Sepuluh Nopember Institute of Technology (ITS) through the written test of science and technology, using Bernoulli Mixture Model in order to evaluate the new student acceptance status. BMM distribution was established based on the comparisonbetween the students’ scores of the basic abilities, namelyMathematics, Physics, Chemistry, and Biology which correspondedto the majors they had chosen, combined with the Student Acceptance Status (0 and 1). This combination generated two components of Mixture, namely right or wrong. The characteristics of each component were then identified through BMM by involving the covariates of Student Acceptance Status, namely the basic ability test and the scholastic test. The combination of Markov Chain Monte Carlo with the Gibbs Sampling algorithm was employed to estimate the parameters used in this research. This method was applied to the data of prospective students who registered in ITS through written test of science and technology. This research result showed the estimated parameters and the formed model of BMM.

PPanGGOLiN: Depicting microbial diversity via a partitioned pangenome graph.

The use of comparative genomics for functional, evolutionary, and epidemiological studies requires methods to classify gene families in terms of occurrence in a given species. These methods usually lack multivariate statistical models to infer the partitions and the optimal number of classes and don't account for genome organization. We introduce a graph structure to model pangenomes in which nodes represent gene families and edges represent genomic neighborhood. Our method, named PPanGGOLiN, partitions nodes using an Expectation-Maximization algorithm based on multivariate Bernoulli Mixture Model coupled with a Markov Random Field. This approach takes into account the topology of the graph and the presence/absence of genes in pangenomes to classify gene families into persistent, cloud, and one or several shell partitions. By analyzing the partitioned pangenome graphs of isolate genomes from 439 species and metagenome-assembled genomes from 78 species, we demonstrate that our method is effective in estimating the persistent genome. Interestingly, it shows that the shell genome is a key element to understand genome dynamics, presumably because it reflects how genes present at intermediate frequencies drive adaptation of species, and its proportion in genomes is independent of genome size. The graph-based approach proposed by PPanGGOLiN is useful to depict the overall genomic diversity of thousands of strains in a compact structure and provides an effective basis for very large scale comparative genomics. The software is freely available at https://github.com/labgem/PPanGGOLiN.

LCBM: A Multi-View Probabilistic Model for Multi-Label Classification.

Multi-label classification is an important research topic in machine learning, for which exploiting label dependencies is an effective modeling principle. Recently, probabilistic models have shown great potential in discovering dependencies among labels. In this paper, motivated by the recent success of multi-view learning to improve the generalization performance, we propose a novel multi-view probabilistic model named latent conditional Bernoulli mixture (LCBM) for multi-label classification. LCBM is a generative model taking features from different views as inputs, and conditional on the latent subspace shared by the views a Bernoulli mixture model is adopted to build label dependencies. Inside each component of the mixture, the labels have a weak correlation which facilitates computational convenience. The mean field variational inference framework is used to carry out approximate posterior inference in the probabilistic model, where we propose a Gaussian mixture variational autoencoder (GMVAE) for effective posterior approximation. We further develop a scalable stochastic training algorithm for efficiently optimizing the model parameters and variational parameters, and derive an efficient prediction procedure based on greedy search. Experimental results on multiple benchmark datasets show that our approach outperforms other state-of-the-art methods under various metrics.

Reliable clustering of Bernoulli mixture models

A Bernoulli Mixture Model (BMM) is a finite mixture of random binary vectors with independent dimensions. The problem of clustering BMM data arises in a variety of real-world applications, ranging from population genetics to activity analysis in social networks. In this paper, we analyze the clusterability of BMMs from a theoretical perspective, when the number of clusters is unknown. In particular, we stipulate a set of conditions on the sample complexity and dimension of the model in order to guarantee the Probably Approximately Correct (PAC)-clusterability of a dataset. To the best of our knowledge, these findings are the first non-asymptotic bounds on the sample complexity of learning or clustering BMMs.

Machine Learning Detectors for MU-MIMO Systems With One-Bit ADCs

We consider an uplink multiuser multiple-input multiple-output (MU-MIMO) system with one-bit analog-to-digital converters (ADCs). In this system, the construction of a low-complexity detector is quite challenging due to the non-linearity of an end-to-end channel transfer function. Recently, a supervised-learning (SL) detector was proposed by modeling the complex non-linear function as a tractable Bernoulli-mixture model. It achieves an optimal maximum-likelihood (ML) performance, provided the channel state information (CSI) is perfectly known at a receiver. However, when a system-size is large, SL detector is not practical because of requiring a large amount of labeled data (i.e., pilot signals) to estimate the model parameters. We address this problem by proposing a semi-supervised learning (SSL) detector in which both pilot signals (i.e., labeled data) and some part of data signals (i.e., unlabeled data) are used to estimate them via expectation-maximization (EM) algorithm. We further extend the proposed detector for time-varying channels, by leveraging the idea of online learning, which is called online-learning (OL) detector. Simulation results demonstrate that the proposed SSL detector can achieve the almost same performance of the corresponding SL detector with significantly lower pilot overhead. In addition, it is shown that the proposed OL detector is more robust to channel variations compared with the existing detectors.

Model Description of Similarity-Based Recommendation Systems.

The quality of online services highly depends on the accuracy of the recommendations they can provide to users. Researchers have proposed various similarity measures based on the assumption that similar people like or dislike similar items or people, in order to improve the accuracy of their services. Additionally, statistical models, such as the stochastic block models, have been used to understand network structures. In this paper, we discuss the relationship between similarity-based methods and statistical models using the Bernoulli mixture models and the expectation-maximization (EM) algorithm. The Bernoulli mixture model naturally leads to a completely positive matrix as the similarity matrix. We prove that most of the commonly used similarity measures yield completely positive matrices as the similarity matrix. Based on this relationship, we propose an algorithm to transform the similarity matrix to the Bernoulli mixture model. Such a correspondence provides a statistical interpretation to similarity-based methods. Using this algorithm, we conduct numerical experiments using synthetic data and real-world data provided from an online dating site, and report the efficiency of the recommendation system based on the Bernoulli mixture models.

On The Bernoulli Mixture Model for Bidikmisi Scholarship Classification with Bayesian MCMC

This research has a purpose to develop Bernoulli Mixture model for Bidikmisi data modelling using Bayesian approach. Model development is done by considering the specificity in the data acceptance of Bidikmisi scholarship prototype in East Java Province. Bidikmisi acceptance status having a binary type (0 and 1) coupled with the main criteria factor of parent income and the number of dependents family produces a structure of Bernoulli mixture distribution with two components. The characteristics of each component can be identified through the Bernoulli Mixture modelling by involving the covariates of Bidikmisi scholarship recipients. The estimating parameter was performed using Bayesian Markov Chain Monte Carlo (MCMC) couple with the Gibbs Sampling algorithm. This model is applied to data registrants Bidikmisi districts/cities in the province of East Java as many as 44,489 students. This model shows the smallest value of Deviance Information Criteria (DIC) compared with Bayesian binary logistic regression.

Bayesian Robust Attributed Graph Clustering: Joint Learning of Partial Anomalies and Group Structure

We study the problem of robust attributed graph clustering. In real data, the clustering structure is often obfuscated due to anomalies or corruptions. While robust methods have been recently introduced that handle anomalies as part of the clustering process, they all fail to account for one core aspect: Since attributed graphs consist of two views (network structure and attributes) anomalies might materialize only partially, i.e. instances might be corrupted in one view but perfectly fit in the other. In this case, we can still derive meaningful cluster assignments. Existing works only consider complete anomalies. In this paper, we present a novel probabilistic generative model (PAICAN) that explicitly models partial anomalies by generalizing ideas of Degree Corrected Stochastic Block Models and Bernoulli Mixture Models. We provide a highly scalable variational inference approach with runtime complexity linear in the number of edges. The robustness of our model w.r.t. anomalies is demonstrated by our experimental study, outperforming state-of-the-art competitors.

Clustering sparse binary data with hierarchical Bayesian Bernoulli mixture model

Sparsity in features presents a big technical challenge to existing clustering methods for categorical data. Hierarchical Bayesian Bernoulli mixture model (HBBMM) incorporates constrained empirical Bayes priors for model parameters, so the resulting Expectation Maximization (EM) algorithm of estimator searching is confined in a proper region. The EM algorithm enables to obtain the maximum a posterior (MAP) estimation, in which cluster labels are simultaneously assigned. Three criteria are proposed to identify defining features of individual clusters, leading to understanding of the underlying data structures. Information based model selection criterion is applied to determine the number of clusters. Estimation consistency and performance of model selection criteria are investigated. Two real-world sparse categorical datasets are analyzed with the proposed method.

Probabilistic analysis of dynamic malware traces

We propose a method to automatically group unknown binaries executed in sandbox according to their interaction with system resources (files on the filesystem, mutexes, registry keys, network communication with remote servers and error messages generated by operating system) such that each group corresponds to a malware family. The method utilizes probabilistic generative model (Bernoulli mixture model), which allows human-friendly prioritization of identified clusters and extraction of readable behavioral indicators to maximize interpretability. We compare it to relevant prior art on a large set of malware binaries where a quality of cluster prioritization and automatic extraction of indicators of compromise is demonstrated. The proposed approach therefore implements complete pipeline which has the potential to significantly speed-up analysis of unknown samples.

Rule-Based Classifier Using Bernoulli Mixture Model for Utilizing Published Rules

Rule-Based Classifier Using Bernoulli Mixture Model for Utilizing Published Rules