Multinomial Mixture Model Research Articles

Characterizing Twitter Discussions About HPV Vaccines Using Topic Modeling and Community Detection.

BackgroundIn public health surveillance, measuring how information enters and spreads through online communities may help us understand geographical variation in decision making associated with poor health outcomes.ObjectiveOur aim was to evaluate the use of community structure and topic modeling methods as a process for characterizing the clustering of opinions about human papillomavirus (HPV) vaccines on Twitter.MethodsThe study examined Twitter posts (tweets) collected between October 2013 and October 2015 about HPV vaccines. We tested Latent Dirichlet Allocation and Dirichlet Multinomial Mixture (DMM) models for inferring topics associated with tweets, and community agglomeration (Louvain) and the encoding of random walks (Infomap) methods to detect community structure of the users from their social connections. We examined the alignment between community structure and topics using several common clustering alignment measures and introduced a statistical measure of alignment based on the concentration of specific topics within a small number of communities. Visualizations of the topics and the alignment between topics and communities are presented to support the interpretation of the results in context of public health communication and identification of communities at risk of rejecting the safety and efficacy of HPV vaccines.ResultsWe analyzed 285,417 Twitter posts (tweets) about HPV vaccines from 101,519 users connected by 4,387,524 social connections. Examining the alignment between the community structure and the topics of tweets, the results indicated that the Louvain community detection algorithm together with DMM produced consistently higher alignment values and that alignments were generally higher when the number of topics was lower. After applying the Louvain method and DMM with 30 topics and grouping semantically similar topics in a hierarchy, we characterized 163,148 (57.16%) tweets as evidence and advocacy, and 6244 (2.19%) tweets describing personal experiences. Among the 4548 users who posted experiential tweets, 3449 users (75.84%) were found in communities where the majority of tweets were about evidence and advocacy.ConclusionsThe use of community detection in concert with topic modeling appears to be a useful way to characterize Twitter communities for the purpose of opinion surveillance in public health applications. Our approach may help identify online communities at risk of being influenced by negative opinions about public health interventions such as HPV vaccines.

Temporal Stability in Chronic Wound Microbiota Is Associated With Poor Healing

Microbial burden of chronic wounds is believed to play an important role in impaired healing and the development of infection-related complications. However, clinical cultures have little predictive value of wound outcomes, and culture-independent studies have been limited by cross-sectional design and small cohort size. We systematically evaluated the temporal dynamics of the microbiota colonizing diabetic foot ulcers, a common and costly complication of diabetes, and its association with healing and clinical complications. Dirichlet multinomial mixture modeling, Markov chain analysis, and mixed-effect models were used to investigate shifts in the microbiota over time and their associations with healing. Here we show, to our knowledge, previously unreported temporal dynamics of the chronic wound microbiome. Microbiota community instability was associated with faster healing and improved outcomes. Diabetic foot ulcer microbiota were found to exist in one of four community types that experienced frequent and nonrandom transitions. Transition patterns and frequencies were associated with healing time. Exposure to systemic antibiotics destabilized the wound microbiota, rather than altering overall diversity or relative abundance of specific taxa. This study provides evidence that the dynamic wound microbiome is indicative of clinical outcomes and may be a valuable guide for personalized management and treatment of chronic wounds.

Fuzzy Co-Clustering Induced by Multinomial Mixture Models

A close connection between fuzzyc-means (FCM) and Gaussian mixture models (GMMs) have been discussed and several extended FCM algorithms were induced by the GMMs concept, where fuzzy partitions are proved to be more useful for revealing intrinsic cluster structures than probabilistic ones. Co-clustering is a promising technique for summarizing cooccurrence information such as document-keyword frequencies. In this paper, a fuzzy co-clustering model is induced based on the multinomial mixture models (MMMs) concept, in which the degree of fuzziness of both object and item fuzzy memberships can be properly tuned. The advantages of the dual fuzzy partition are demonstrated through several experimental results including document clustering applications.

Boosting Response Aware Model-Based Collaborative Filtering

Recommender systems are promising for providing personalized favorite services. Collaborative filtering (CF) technologies, making prediction of users’ preference based on users’ previous behaviors, have become one of the most successful techniques to build modern recommender systems. Several challenging issues occur in previously proposed CF methods: 1) most CF methods ignore users’ response patterns and may yield biased parameter estimation and suboptimal performance; 2) some CF methods adopt heuristic weight settings, which lacks a systematical implementation; and 3) the multinomial mixture models may weaken the computational ability of matrix factorization for generating the data matrix, thus increasing the computational cost of training. To resolve these issues, we incorporate users’ response models into the probabilistic matrix factorization (PMF), a popular matrix factorization CF model, to establish the response aware probabilistic matrix factorization (RAPMF) framework. More specifically, we make the assumption on the user response as a Bernoulli distribution which is parameterized by the rating scores for the observed ratings while as a step function for the unobserved ratings. Moreover, we speed up the algorithm by a mini-batch implementation and a crafting scheduling policy. Finally, we design different experimental protocols and conduct systematical empirical evaluation on both synthetic and real-world datasets to demonstrate the merits of the proposed RAPMF and its mini-batch implementation.

Structure of the gut microbiome following colonization with human feces determines colonic tumor burden

BackgroundA growing body of evidence indicates that the gut microbiome plays a role in the development of colorectal cancer (CRC). Patients with CRC harbor gut microbiomes that are structurally distinct from those of healthy individuals; however, without the ability to track individuals during disease progression, it has not been possible to observe changes in the microbiome over the course of tumorigenesis. Mouse models have demonstrated that these changes can further promote colonic tumorigenesis. However, these models have relied upon mouse-adapted bacterial populations and so it remains unclear which human-adapted bacterial populations are responsible for modulating tumorigenesis.ResultsWe transplanted fecal microbiota from three CRC patients and three healthy individuals into germ-free mice, resulting in six structurally distinct microbial communities. Subjecting these mice to a chemically induced model of CRC resulted in different levels of tumorigenesis between mice. Differences in the number of tumors were strongly associated with the baseline microbiome structure in mice, but not with the cancer status of the human donors. Partitioning of baseline communities into enterotypes by Dirichlet multinomial mixture modeling resulted in three enterotypes that corresponded with tumor burden. The taxa most strongly positively correlated with increased tumor burden were members of the Bacteroides, Parabacteroides, Alistipes, and Akkermansia, all of which are Gram-negative. Members of the Gram-positive Clostridiales, including multiple members of Clostridium Group XIVa, were strongly negatively correlated with tumors. Analysis of the inferred metagenome of each community revealed a negative correlation between tumor count and the potential for butyrate production, and a positive correlation between tumor count and the capacity for host glycan degradation. Despite harboring distinct gut communities, all mice underwent conserved structural changes over the course of the model. The extent of these changes was also correlated with tumor incidence.ConclusionOur results suggest that the initial structure of the microbiome determines susceptibility to colonic tumorigenesis. There appear to be opposing roles for certain Gram-negative (Bacteroidales and Verrucomicrobia) and Gram-positive (Clostridiales) bacteria in tumor susceptibility. Thus, the impact of community structure is potentially mediated by the balance between protective, butyrate-producing populations and inflammatory, mucin-degrading populations.

Discriminative learning of generative models: large margin multinomial mixture models for document classification

In this paper, a novel discriminative learning method is proposed to estimate generative models for multi-class pattern classification tasks, where a discriminative objective function is formulated with separation margins according to certain discriminative learning criterion, such as large margin estimation (LME). Furthermore, the so-called approximation-maximization (AM) method is proposed to optimize the discriminative objective function w.r.t. parameters of generative models. The AM approach provides a good framework to deal with latent variables in generative models and it is flexible enough to discriminatively learn many rather complicated generative models. In this paper, we are interested in a group of generative models derived from multinomial distributions. Under some minor relaxation conditions, it is shown that the AM-based discriminative learning methods for these generative models result in linear programming (LP) problems that can be solved effectively and efficiently even for rather large-scale models. As a case study, we have studied to learn multinomial mixture models (MMMs) for text document classification based on the large margin criterion. The proposed methods have been evaluated on a standard RCV1 text corpus. Experimental results show that large margin MMMs significantly outperform the conventional MMMs as well as pure discriminative models such as support vector machines (SVM), where over 25 % relative classification error reduction is observed in three independent RCV1 test sets.

Dynamics and associations of microbial community types across the human body.

A primary goal of the Human Microbiome Project (HMP) was to provide a reference collection of 16S ribosomal RNA gene sequences collected from sites across the human body that would allow microbiologists to better associate changes in the microbiome with changes in health. The HMP Consortium has reported the structure and function of the human microbiome in 300 healthy adults at 18 body sites from a single time point. Using additional data collected over the course of 12-18 months, we used Dirichlet multinomial mixture models to partition the data into community types for each body site and made three important observations. First, there were strong associations between whether individuals had been breastfed as an infant, their gender, and their level of education with their community types at several body sites. Second, although the specific taxonomic compositions of the oral and gut microbiomes were different, the community types observed at these sites were predictive of each other. Finally, over the course of the sampling period, the community types from sites within the oral cavity were the least stable, whereas those in the vagina and gut were the most stable. Our results demonstrate that even with the considerable intra- and interpersonal variation in the human microbiome, this variation can be partitioned into community types that are predictive of each other and are probably the result of life-history characteristics. Understanding the diversity of community types and the mechanisms that result in an individual having a particular type or changing types, will allow us to use their community types to assess disease risk and to personalize therapies.

A Bayesian cluster analysis of election results

A Bayesian cluster analysis for the results of an election based on multinomial mixture models is proposed. The number of clusters is chosen based on the careful comparison of the results with predictive simulations from the models, and by checking whether models capture most of the spatial dependence in the results. By implementing the analysis on five recent elections in Barcelona, the reader is walked through the choice of the best statistics and graphical displays to help chose a model and present the results. Even though the models do not use any information about the location of the areas in which the results are broken into, in the example they uncover a four-cluster structure with a strong spatial dependence, that is very stable over time and relates to the demographic composition.

Malaria haplotype frequency estimation

We present a Bayesian approach for estimating the relative frequencies of multi-single nucleotide polymorphism (SNP) haplotypes in populations of the malaria parasite Plasmodium falciparum by using microarray SNP data from human blood samples. Each sample comes from a malaria patient and contains one or several parasite clones that may genetically differ. Samples containing multiple parasite clones with different genetic markers pose a special challenge. The situation is comparable with a polyploid organism. The data from each blood sample indicates whether the parasites in the blood carry a mutant or a wildtype allele at various selected genomic positions. If both mutant and wildtype alleles are detected at a given position in a multiply infected sample, the data indicates the presence of both alleles, but the ratio is unknown. Thus, the data only partially reveals which specific combinations of genetic markers (i.e. haplotypes across the examined SNPs) occur in distinct parasite clones. In addition, SNP data may contain errors at non-negligible rates. We use a multinomial mixture model with partially missing observations to represent this data and a Markov chain Monte Carlo method to estimate the haplotype frequencies in a population. Our approach addresses both challenges, multiple infections and data errors.

Clustering and variable selection for categorical multivariate data

This article investigates unsupervised classification techniques for categorical multivariate data. The study employs multivariate multinomial mixture modeling, which is a type of model particularly applicable to multilocus genotypic data. A model selection procedure is used to simultaneously select the number of components and the relevant variables. A non-asymptotic oracle inequality is obtained, leading to the proposal of a new penalized maximum likelihood criterion. The selected model proves to be asymptotically consistent under weak assumptions on the true probability underlying the observations. The main theoretical result obtained in this study suggests a penalty function defined to within a multiplicative parameter. In practice, the data-driven calibration of the penalty function is made possible by slope heuristics. Based on simulated data, this procedure is found to improve the performance of the selection procedure with respect to classical criteria such as BIC and AIC. The new criterion provides an answer to the question "Which criterion for which sample size?" Examples of real dataset applications are also provided.

Hierarchical modelling and estimation of abundance and population trends in metapopulation designs

1. Population assessment in changing environments is challenging because factors governing abundance may also affect detectability and thus bias observed counts. We describe a hierarchical modelling framework for estimating abundance corrected for detectability in metapopulation designs, where observations of 'individuals' (e.g. territories) are replicated in space and time. We consider two classes of models; first, we regard the data as independent binomial counts and model abundance and detectability based on a product-binomial likelihood. Secondly, we use the more complex detection-non-detection data for each territory to form encounter history frequencies, and analyse the resulting multinomial/Poisson hierarchical model. Importantly, we extend both models to directly estimate population trends over multiple years. Our models correct for any time trends in detectability when assessing population trends in abundance. 2. We illustrate both models for a farmland and a woodland bird species, skylark Alauda arvensis and willow tit Parus montanus, by applying them to Swiss BBS data, where 268 1 km(2) quadrats were surveyed two to three times during 1999-2003. We fit binomial and multinomial mixture models where log(abundance) depended on year, elevation, forest cover and transect route length, and logit(detection) on year, season and search effort. 3. Parameter estimates were very similar between models with confidence intervals overlapping for most parameters. Trend estimates were similar for skylark (-0.074 +/- 0.041 vs. -0.047 +/- 0.019) and willow tit (0.044 +/- 0.046 vs. 0.047 +/- 0.018). As expected, the multinomial model gave more precise estimates, but also yielded lower abundance estimates for the skylark. This may be due to effects of territory misclassification (lumping error), which do not affect the binomial model. 4. Both models appear useful for estimating abundance and population trends free from distortions by detectability in metapopulation designs with temporally replicated observations. The ability to obtain estimates of abundance and population trends that are unbiased with respect to any time trends in detectability ought to be a strong motivation for the collection of replicate observation data.

Multinomial mixture model with heterogeneous classification probabilities

Royle and Link (Ecology 86(9):2505–2512, 2005) proposed an analytical method that allowed estimation of multinomial distribution parameters and classification probabilities from categorical data measured with error. While useful, we demonstrate algebraically and by simulations that this method yields biased multinomial parameter estimates when the probabilities of correct category classifications vary among sampling units. We address this shortcoming by treating these probabilities as logit-normal random variables within a Bayesian framework. We use Markov chain Monte Carlo to compute Bayes estimates from a simulated sample from the posterior distribution. Based on simulations, this elaborated Royle-Link model yields nearly unbiased estimates of multinomial and correct classification probability estimates when classification probabilities are allowed to vary according to the normal distribution on the logit scale or according to the Beta distribution. The method is illustrated using categorical submersed aquatic vegetation data.

Improving document clustering in a learned concept space

Most document clustering algorithms operate in a high dimensional bag-of-words space. The inherent presence of noise in such representation obviously degrades the performance of most of these approaches. In this paper we investigate an unsupervised dimensionality reduction technique for document clustering. This technique is based upon the assumption that terms co-occurring in the same context with the same frequencies are semantically related. On the basis of this assumption we first find term clusters using a classification version of the EM algorithm. Documents are then represented in the space of these term clusters and a multinomial mixture model ( MM) is used to build document clusters. We empirically show on four document collections, Reuters-21578, Reuters RCV2-French, 20Newsgroups and WebKB, that this new text representation noticeably increases the performance of the MM model. By relating the proposed approach to the Probabilistic Latent Semantic Analysis ( PLSA) model we further propose an extension of the latter in which an extra latent variable allows the model to co-cluster documents and terms simultaneously. We show on these four datasets that the proposed extended version of the PLSA model produces statistically significant improvements with respect to two clustering measures over all variants of the original PLSA and the MM models.

A discrete mixture-based kernel for SVMs: Application to spam and image categorization

In this paper, we investigate the problem of training support vector machines (SVMs) on count data. Multinomial Dirichlet mixture models allow us to model efficiently count data. On the other hand, SVMs permit good discrimination. We propose, then, a hybrid model that appropriately combines their advantages. Finite mixture models are introduced, as an SVM kernel, to incorporate prior knowledge about the nature of data involved in the problem at hand. For the learning of our mixture model, we propose a deterministic annealing component-wise EM algorithm mixed with a minimum description length type criterion. In the context of this model, we compare different kernels. Through some applications involving spam and image database categorization, we find that our data-driven kernel performs better.

Match Play: Using Statistical Methods to Categorize PGA Tour Players' Careers

This paper uses K-means cluster analysis and multinomial mixture models to categorize professional golfer performance for the period 1980 to 2006. We collected, cleaned and analyzed final annual money list standings for PGA Tour players from www.pgatour.com. Correlation patterns between other measures suggested that career performance was well described by the proportion of years a player finished in specific meaningful money list categories such as Top 10 or outside the top 125. Using both clustering methods, we found that players divide into five natural and interpretable groupings, one being a small `Elite' group and four others which we refer to as `Distinguished,' `Established,' `Journeymen' and `Grinders.' We used analysis of variance to compare groups on the basis of other career performance measures including consistency, streakiness, longevity, and others as well as to investigate differences in the clusters produced by the two methods. A summary of these measures for the entire set gives a bleak outlook for any prospective professional golfer and singles out the greatness of the elite few, epitomized by Tiger Woods. This methodology extends to any sport or endeavor in which performance is measured in terms of end of year rankings or money lists.

Clustering Discrete Data Through the Multinomial Mixture Model

In this work, the multinomial mixture model is studied, through a maximum likelihood approach. The convergence of the maximum likelihood estimator to a set with characteristics of interest is shown. A method to select the number of mixture components is developed based on the form of the maximum likelihood estimator. A simulation study is then carried out to verify its behavior. Finally, two applications on real data of multinomial mixtures are presented.

Complexity control in a mixture model by the Hardy–Weinberg equilibrium

A method of complexity control in multinomial mixture modeling of multiple-marker genotype data, imposing the Hardy–Weinberg equilibrium (HWE) between the genotype values, is studied. This is a very natural restriction, and known to hold at population level under modest assumptions. The hypothesis under study is that imposing this restriction will prevent overfitting and lead to a better model. This is shown to indeed be case. Experimental results on chromosomes 1 and 17 of the HapMap data demonstrate that the restricted model generalizes better to unseen data, and also finds clusters that correspond better to the ethnic groups of the HapMap, when compared with a model without the HWE restriction.

Multinomial mixture model with feature selection for text clustering

The task of selecting relevant features is a hard problem in the field of unsupervised text clustering due to the absence of class labels that would guide the search. This paper proposes a new mixture model method for unsupervised text clustering, named multinomial mixture model with feature selection (M3FS). In M3FS, we introduce the concept of component-dependent “feature saliency” to the mixture model. We say a feature is relevant to a certain mixture component if the feature saliency value is higher than a predefined threshold. Thus the feature selection process is treated as a parameter estimation problem. The Expectation–Maximization (EM) algorithm is then used for estimating the model. The experiment results on commonly used text datasets show that the M3FS method has good clustering performance and feature selection capability.

Mixture Modeling for Genome‐Wide Localization of Transcription Factors

Chromatin immunoprecipitation followed by DNA microarray analysis (ChIP-chip methodology) is an efficient way of mapping genome-wide protein-DNA interactions. Data from tiling arrays encompass DNA-protein interaction measurements on thousands or millions of short oligonucleotides (probes) tiling a whole chromosome or genome. We propose a new model-based method for analyzing ChIP-chip data. The proposed model is motivated by the widely used two-component multinomial mixture model of de novo motif finding. It utilizes a hierarchical gamma mixture model of binding intensities while incorporating inherent spatial structure of the data. In this model, genomic regions belong to either one of the following two general groups: regions with a local protein-DNA interaction (peak) and regions lacking this interaction. Individual probes within a genomic region are allowed to have different localization rates accommodating different binding affinities. A novel feature of this model is the incorporation of a distribution for the peak size derived from the experimental design and parameters. This leads to the relaxation of the fixed peak size assumption that is commonly employed when computing a test statistic for these types of spatial data. Simulation studies and a real data application demonstrate good operating characteristics of the method including high sensitivity with small sample sizes when compared to available alternative methods.

Supervised Detection of Conserved Motifs in DNA Sequences with Cosmo

A number of computational methods have been proposed for identifying transcription factor binding sites from a set of unaligned sequences that are thought to share the motif in question. We here introduce an algorithm, called cosmo, that allows this search to be supervised by specifying a set of constraints that the position weight matrix of the unknown motif must satisfy. Such constraints may be formulated, for example, on the basis of prior knowledge about the structure of the transcription factor in question. The algorithm is based on the same two-component multinomial mixture model used by MEME, with stronger reliance, however, on the likelihood principle instead of more ad-hoc criteria like the E-value. The intensity parameter in the ZOOPS and TCM models, for instance, is estimated based on a profile-likelihood approach, and the width of the unknown motif is selected based on BIC. These changes allow cosmo to outperform MEME even in the absence of any constraints, as evidenced by 2- to 3-fold greater sensitivity in some simulation studies. Additional improvements in performance can be achieved by selecting the model type (OOPS, ZOOPS, or TCM) data-adaptively or by supplying correctly specified constraints, especially if the motif appears only as a weak signal in the data. The algorithm can data-adaptively choose between working in a given constrained model or in the completely unconstrained model, guarding against the risk of supplying mis-specified constraints. Simulation studies suggest that this approach can offer 3 to 3.5 times greater sensitivity than MEME. The algorithm has been implemented in the form of a stand-alone C program as well as a web application that can be accessed at http://cosmoweb.berkeley.edu. An R package is available through Bioconductor (http://bioconductor.org).