Dirichlet Multinomial Mixture Model Research Articles

Hybrid topic modeling method based on dirichlet multinomial mixture and fuzzy match algorithm for short text clustering

Topic modeling methods proved to be effective for inferring latent topics from short texts. Dealing with short texts is challenging yet helpful for many real-world applications, due to the sparse terms in the text and the high dimensionality representation. Most of the topic modeling methods require the number of topics to be defined earlier. Similarly, methods based on Dirichlet Multinomial Mixture (DMM) involve the maximum possible number of topics before execution which is hard to determine due to topic uncertainty, and many noises exist in the dataset. Hence, a new approach called the Topic Clustering algorithm based on Levenshtein Distance (TCLD) is introduced in this paper, TCLD combines DMM models and the Fuzzy matching algorithm to address two key challenges in topic modeling: (a) The outlier problem in topic modeling methods. (b) The problem of determining the optimal number of topics. TCLD uses the initial clustered topics generated by DMM models and then evaluates the semantic relationships between documents using Levenshtein Distance. Subsequently, it determines whether to keep the document in the same cluster, relocate it to another cluster, or mark it as an outlier. The results demonstrate the efficiency of the proposed approach across six English benchmark datasets, in comparison to seven topic modeling approaches, with 83% improvement in purity and 67% enhancement in Normalized Mutual Information (NMI) across all datasets. The proposed method was also applied to a collected Arabic tweet and the results showed that only 12% of the Arabic short texts were incorrectly clustered, according to human inspection.

Integrating semantic similarity with Dirichlet multinomial mixture model for enhanced web service clustering

Integrating semantic similarity with Dirichlet multinomial mixture model for enhanced web service clustering

Online short text clustering using infinite extensions of discrete mixture models

AbstractShort text clustering is one of the fundamental tasks in natural language processing. Different from traditional documents, short texts are ambiguous and sparse due to their short form and the lack of recurrence in word usage from one text to another, making it very challenging to apply conventional machine learning algorithms directly. In this article, we propose two novel approaches for short texts clustering: collapsed Gibbs sampling infinite generalized Dirichlet multinomial mixture model infinite GSGDMM) and collapsed Gibbs sampling infinite Beta‐Liouville multinomial mixture model (infinite GSBLMM). We adopt two flexible and practical priors to the multinomial distribution where in the first one the generalized Dirichlet distribution is integrated, while the second one is based on the Beta‐Liouville distribution. We evaluate the proposed approaches on two famous benchmark datasets, namely, Google News and Tweet. The experimental results demonstrate the effectiveness of our models compared to basic approaches that use Dirichlet priors. We further propose to improve the performance of our methods with an online clustering procedure. We also evaluate the performance of our methods for the outlier detection task, in which we achieve accurate results.

Gut Microbiota Enterotypes Mediate the Effects of Dietary Patterns on Colorectal Neoplasm Risk in a Chinese Population.

Colorectal cancer (CRC) risk is influenced by dietary patterns and gut microbiota enterotypes. However, the interaction between these factors remains unclear. This study examines this relationship, hypothesizing that different diets may affect colorectal tumor risk in individuals with varied gut microbiota enterotypes. We conducted a case-control study involving 410 Han Chinese individuals, using exploratory structural equation modeling to identify two dietary patterns, and a Dirichlet multinomial mixture model to classify 250 colorectal neoplasm cases into three gut microbiota enterotypes. We assessed the association between dietary patterns and the risk of each tumor subtype using logistic regression analysis. We found that a healthy diet, rich in vegetables, fruits, milk, and yogurt, lowers CRC risk, particularly in individuals with type I (dominated by Bacteroides and Lachnoclostridium) and type II (dominated by Bacteroides and Faecalibacterium) gut microbiota enterotypes, with adjusted odds ratios (ORs) of 0.66 (95% confidence interval [CI] = 0.48-0.89) and 0.42 (95% CI = 0.29-0.62), respectively. Fruit consumption was the main contributor to this protective effect. No association was found between a healthy dietary pattern and colorectal adenoma risk or between a high-fat diet and colorectal neoplasm risk. Different CRC subtypes associated with gut microbiota enterotypes displayed unique microbial compositions and functions. Our study suggests that specific gut microbiota enterotypes can modulate the effects of diet on CRC risk, offering new perspectives on the relationship between diet, gut microbiota, and colorectal neoplasm risk.

WGSDMM+GA: A genetic algorithm-based service clustering methodology assimilating dirichlet multinomial mixture model with word embedding

Advancement of web 2.0 results in the expeditious growth of services in repositories and service portals, which raises the demand for service management. With the clusters of services, different processes like discovery, selection, ranking, etc., can be expedited efficiently. K-Means clustering is highly recommended for service clustering but it can be stuck in the local optimum values. Developers generally use short text to describe the functionality of Web services. For effective service clustering, it is required that relevant features of services are mapped in vector space based on semantic relations. Gibbs Sampling algorithm for Dirichlet Multinomial Mixture model (GSDMM) model is able to provide good performance on short text because it works on the assumption that there will be one topic related to one document. In this paper, a novel technique for web service clustering, i.e., WGSDMM+GA, is proposed in which relevant features, semantic relations, and their mappings in vector space are analyzed based on an assimilated approach of Word2Vec and GSDMM models. As K-means clustering cannot provide effective results due to local optima problem, GA-based clustering is applied in which fitness function is based on Manhattan distance. The proposed model is applied to five real-time datasets, and the performance is evaluated based on standard evaluation measures. Experimental results illustrate that WGSDMM+GA yields better results, and the accuracy score is increased by 23%, 17%,17%, 21%, and 25% on five real-time datasets.

Sparse tree-based clustering of microbiome data to characterize microbiome heterogeneity in pancreatic cancer.

There is a keen interest in characterizing variation in the microbiome across cancer patients, given increasing evidence of its important role in determining treatment outcomes. Here our goal is to discover subgroups of patients with similar microbiome profiles. We propose a novel unsupervised clustering approach in the Bayesian framework that innovates over existing model-based clustering approaches, such as the Dirichlet multinomial mixture model, in three key respects: we incorporate feature selection, learn the appropriate number of clusters from the data, and integrate information on the tree structure relating the observed features. We compare the performance of our proposed method to existing methods on simulated data designed to mimic real microbiome data. We then illustrate results obtained for our motivating data set, a clinical study aimed at characterizing the tumor microbiome of pancreatic cancer patients.

Topic modelling and emotion analysis of the tweets of British and American politicians on the topic of war in Ukraine

This paper focuses on the content and emotive features of four politicians' posts that were published on their official Twitter accounts during the three-month period of the russian invasion of Ukraine. We selected two British politicians – Boris Johnson, the Prime Minister of the UK, and Yvette Cooper, the Labour MP and Shadow Home Secretary of the State for the Home Department – as well as two American politicians, President Joe Biden and Republican senator Marco Rubio. In the first phase, we identified the most frequent lexical tokens used by the politicians to inform the world community about the war in Ukraine. For this purpose, we used Voyant Tools, a web-based application for text analysis. These tokens were divided into three groups according to the level of their frequency. Additionally, we measured the distribution of the most frequent lexical tokens across the three-month time span. In the next phase, we analysed the context of the identified lexical tokens, thereby outlining the subject of the tweets. To do this, we extracted collocations using the Natural Language Toolkit (NLTK) library. During the final phase of the research, we performed topic modelling using the Gibbs Sampling algorithm for the Dirichlet Multinomial Mixture model (GSDMM) and emotion analysis using the NRC Lexicon library.

Stochastic variational variable selection for high-dimensional microbiome data

BackgroundThe rapid and accurate identification of a minimal-size core set of representative microbial species plays an important role in the clustering of microbial community data and interpretation of clustering results. However, the huge dimensionality of microbial metagenomics datasets is a major challenge for the existing methods such as Dirichlet multinomial mixture (DMM) models. In the approach of the existing methods, the computational burden of identifying a small number of representative species from a large number of observed species remains a challenge.ResultsWe propose a novel approach to improve the performance of the widely used DMM approach by combining three ideas: (i) we propose an indicator variable to identify representative operational taxonomic units that substantially contribute to the differentiation among clusters; (ii) to address the computational burden of high-dimensional microbiome data, we propose a stochastic variational inference, which approximates the posterior distribution using a controllable distribution called variational distribution, and stochastic optimization algorithms for fast computation; and (iii) we extend the finite DMM model to an infinite case by considering Dirichlet process mixtures and estimating the number of clusters as a variational parameter. Using the proposed method, stochastic variational variable selection (SVVS), we analyzed the root microbiome data collected in our soybean field experiment, the human gut microbiome data from three published datasets of large-scale case-control studies and the healthy human microbiome data from the Human Microbiome Project.ConclusionsSVVS demonstrates a better performance and significantly faster computation than those of the existing methods in all cases of testing datasets. In particular, SVVS is the only method that can analyze massive high-dimensional microbial data with more than 50,000 microbial species and 1000 samples. Furthermore, a core set of representative microbial species is identified using SVVS that can improve the interpretability of Bayesian mixture models for a wide range of microbiome studies.5HV9399yjMcY2fvU9ASUoUVideo

Activity discovery using Dirichlet multinomial mixture models from discrete sensor data in smart homes

To develop smart home technology designed to analyze the activity of residents based on the logs of installed sensors, an activity model tailored to individuals must be constructed from less privacy-invasive sensors to avoid interference in daily life. Unsupervised machine learning techniques are desirable to automatically construct such models without costly data annotation, but their application has not yet been sufficiently successful. In this study, we show that an activity model can be effectively estimated without activity labels via the Dirichlet multinomial mixture (DMM) model. The DMM model assumes that sensor signals are generated according to a Dirichlet multinomial distribution conditioned on a single unobservable activity and can capture the burstiness of sensors, in which even sensors that rarely fire may fire repeatedly after being triggered. We demonstrate the burstiness phenomenon in real data using passive infrared ray motion sensors. For such data, the assumptions of the DMM model are more suitable than the assumptions employed in models used in previous studies. Moreover, we extend the DMM model so that each activity depends on the preceding activity to capture the Markov dependency of activities, and a Gibbs sampler used in the model estimation algorithm is also presented. An empirical study using publicly available data collected in real-life settings shows that the DMM models can discover activities more correctly than the other models and expected to be used as a primitive activity extraction tool in activity analysis.

Railway Fault Text Clustering Method Using an Improved Dirichlet Multinomial Mixture Model

Railway signal equipment fault data (RSEFD) are one of the issues with in-depth traffic big data analysis throughout the life cycle of intelligent transportation. In the course of daily operation and maintenance, the railway electrical maintenance department records equipment malfunction information in a natural language. The data have the characteristics of strong professionalism, short text, unbalanced category, and low efficiency of manual analysis and processing. How to effectively mine the information contained in these fault texts to provide help for on-site operation and maintenance plays an important role. Therefore, we propose a railway fault text clustering method using an improved Dirichlet multinomial mixture model called ICH-GSDMM. In this method, first, the railway signal terminology thesaurus is established to overcome the inaccurate problem of RSEFD segmentation. Second, the traditional Chi square statistics is improved to overcome the learning difficulties caused by the imbalance of RSEFD. Finally, the Gibbs sampling algorithm for Dirichlet multinomial mixture model (GSDMM) is modified using an improved chi-square statistical method (ICH) to overcome the symmetry problem of the word Dirichlet prior parameters in the traditional GSDMM. Compared to the traditional GSDMM model and the GSDMM model based on chi-square statistics (CH-GSDMM), the quantitative experimental results show that the GSDMM model based on improved chi-square statistics (ICH-GSDMM internal)’s evaluation index of clustering performance has greatly improved, and its external evaluation indices are also the best, with the exception of external index NMI of data set DS2. Simultaneously, the diagnostic accuracy of a select few categories in RSEFD has considerably improved, demonstrating its efficacy.

ChromDMM: a Dirichlet-multinomial mixture model for clustering heterogeneous epigenetic data.

MotivationResearch on epigenetic modifications and other chromatin features at genomic regulatory elements elucidates essential biological mechanisms including the regulation of gene expression. Despite the growing number of epigenetic datasets, new tools are still needed to discover novel distinctive patterns of heterogeneous epigenetic signals at regulatory elements.ResultsWe introduce ChromDMM, a product Dirichlet-multinomial mixture model for clustering genomic regions that are characterized by multiple chromatin features. ChromDMM extends the mixture model framework by profile shifting and flipping that can probabilistically account for inaccuracies in the position and strand-orientation of the genomic regions. Owing to hyper-parameter optimization, ChromDMM can also regularize the smoothness of the epigenetic profiles across the consecutive genomic regions. With simulated data, we demonstrate that ChromDMM clusters, shifts and strand-orients the profiles more accurately than previous methods. With ENCODE data, we show that the clustering of enhancer regions in the human genome reveals distinct patterns in several chromatin features. We further validate the enhancer clusters by their enrichment for transcriptional regulatory factor binding sites.Availability and implementationChromDMM is implemented as an R package and is available at https://github.com/MariaOsmala/ChromDMM.Supplementary information Supplementary data are available at Bioinformatics online.

A Web service clustering method based on topic enhanced Gibbs sampling algorithm for the Dirichlet Multinomial Mixture model and service collaboration graph

A method to enhance Web service clustering is proposed in this paper. Since current service clustering methods usually face low quality of service representation vectors and lack consideration of service collaboration, we try to provide an improved topic model to generate high-quality service representation vectors and design a service clustering method to integrate function similarity and collaboration similarity. First, by introducing feature word extraction and probability distribution correction into GSDMM, we present a model called TE-GSDMM (topic enhanced Gibbs sampling algorithm for the Dirichlet Multinomial Mixture model). Then, a service collaboration graph is put forward to model cooperation relationships and generate service collaboration vectors. Collaboration similarity is assessed by the similarity of service collaboration vectors. Finally, the K-means++ algorithm is employed to cluster Web services by evaluating service function similarity and collaboration similarity. Experiments show that TE-GSDMM outperforms other topic models in generating high-quality service representation vectors for service clustering. Moreover, service clustering performance is further improved by integrating collaboration similarity. Thus, the proposed method effectively enhances Web service clustering by improving the quality of service representation vectors and integrating service collaboration similarity.

Association of bacterial community types, functional microbial processes and lung disease in cystic fibrosis airways.

Bacterial infection and inflammation of the airways are the leading causes of morbidity and mortality in persons with cystic fibrosis (CF). The ecology of the bacterial communities inhabiting CF airways is poorly understood, especially with respect to how community structure, dynamics, and microbial metabolic activity relate to clinical outcomes. In this study, the bacterial communities in 818 sputum samples from 109 persons with CF were analyzed by sequencing bacterial 16S rRNA gene amplicons. We identified eight alternative community types (pulmotypes) by using a Dirichlet multinomial mixture model and studied their temporal dynamics in the cohort. Across patients, the pulmotypes displayed chronological patterns in the transition among each other. Furthermore, significant correlations between pulmotypes and patient clinical status were detected by using multinomial mixed effects models, principal components regression, and statistical testing. Constructing pulmotype-specific metabolic activity profiles, we found that pulmotype microbiota drive distinct community functions including mucus degradation or increased acid production. These results indicate that pulmotypes are the result of ordered, underlying drivers such as predominant metabolism, ecological competition, and niche construction and can form the basis for quantitative, predictive models supporting clinical treatment decisions.

Automatic Detection of the Topics in Customer Complaints with Artificial Intelligence

Today, people first make their complaints and compliments on internet about a product which they use or a company they are a customer of. Therefore, when they are going to buy a new product, they first analyze the complaints made by other users of the product. These complaints play an important role in helping people make decision of purchasing or not purchasing product. It is impossible to analyze online complaints manually due to the huge data size. However, companies are still losing a lot of time by analyzing and reading thousands of complaints one by one. In this article, online text based customer complaints are analyzed with Latent Dirichlet Allocation (LDA), GenSim LDA, Mallet LDA and Gibbs Sampling for Dirichlet Multinomial Mixture model (GSDMM) and the performances of them are compared. It is observed that GSDMM gives much more successful results than LDA. The obtained topics of the complaints are presented to users with a mobile application developed in React Native. With the developed application not only the customers will be able to see the topics of complaint from the application interface but also the companies will be able to view the distribution and statistics of the topics of complaints.

Destination shapes antibiotic resistance gene acquisitions, abundance increases, and diversity changes in Dutch travelers

BackgroundAntimicrobial-resistant bacteria and their antimicrobial resistance (AMR) genes can spread by hitchhiking in human guts. International travel can exacerbate this public health threat when travelers acquire AMR genes endemic to their destinations and bring them back to their home countries. Prior studies have demonstrated travel-related acquisition of specific opportunistic pathogens and AMR genes, but the extent and magnitude of travel’s effects on the gut resistome remain largely unknown.MethodsUsing whole metagenomic shotgun sequencing, functional metagenomics, and Dirichlet multinomial mixture models, we investigated the abundance, diversity, function, resistome architecture, and context of AMR genes in the fecal microbiomes of 190 Dutch individuals, before and after travel to diverse international locations.ResultsTravel markedly increased the abundance and α-diversity of AMR genes in the travelers’ gut resistome, and we determined that 56 unique AMR genes showed significant acquisition following international travel. These acquisition events were biased towards AMR genes with efflux, inactivation, and target replacement resistance mechanisms. Travel-induced shaping of the gut resistome had distinct correlations with geographical destination, so individuals returning to The Netherlands from the same destination country were more likely to have similar resistome features. Finally, we identified and detailed specific acquisition events of high-risk, mobile genetic element-associated AMR genes including qnr fluoroquinolone resistance genes, blaCTX-M family extended-spectrum β-lactamases, and the plasmid-borne mcr-1 colistin resistance gene.ConclusionsOur results show that travel shapes the architecture of the human gut resistome and results in AMR gene acquisition against a variety of antimicrobial drug classes. These broad acquisitions highlight the putative risks that international travel poses to public health by gut resistome perturbation and the global spread of locally endemic AMR genes.

A reliable cross-site user generated content modeling method based on topic model

Nowadays, social network sites (SNSs) have been significant platforms for content sharing in our daily life. With the emergence of different kinds of social network sites and users’ diverse needs for content sharing, their content sharing practices are generally taken place in multiple SNSs. To construct models that can characterize users’ content sharing practices in a composite context constituted by multiple social network sites (cross-site user generated content modeling) has been an emerging research topic in web data mining and human behavior research. However, previous methods such as Dirichlet Multinomial Mixture model (DMM), Biterm Topic Model (BTM), Twitter-LDA and MultiLDA have limited representation ability or are based on unreliable assumption, which cannot characterize the user generated content (UGC) accurately from the perspective of multiple SNSs. In this paper, we first conduct an empirical study to investigate the characteristics of users’ content sharing practices in cross-site context, based on which we propose a more reliable cross-site UGC model named CrossSite-LDA (C-LDA). We then evaluate the performances of the C-LDA model with four state-of-the-art models based on the two data sets sampled from Weibo–Douban and Facebook–Twitter. Results show that the C-LDA has better performances in perplexity, word coherence, topic KL divergence, UCI and UMass metrics compared with existing models, which suggests its superior accuracy on modeling users’ content characteristics in cross-site context.

A Gamma-Poisson Mixture Topic Model for Short Text

Most topic models are constructed under the assumption that documents follow a multinomial distribution. The Poisson distribution is an alternative distribution to describe the probability of count data. For topic modelling, the Poisson distribution describes the number of occurrences of a word in documents of fixed length. The Poisson distribution has been successfully applied in text classification, but its application to topic modelling is not well documented, specifically in the context of a generative probabilistic model. Furthermore, the few Poisson topic models in the literature are admixture models, making the assumption that a document is generated from a mixture of topics. In this study, we focus on short text. Many studies have shown that the simpler assumption of a mixture model fits short text better. With mixture models, as opposed to admixture models, the generative assumption is that a document is generated from a single topic. One topic model, which makes this one-topic-per-document assumption, is the Dirichlet-multinomial mixture model. The main contributions of this work are a new Gamma-Poisson mixture model, as well as a collapsed Gibbs sampler for the model. The benefit of the collapsed Gibbs sampler derivation is that the model is able to automatically select the number of topics contained in the corpus. The results show that the Gamma-Poisson mixture model performs better than the Dirichlet-multinomial mixture model at selecting the number of topics in labelled corpora. Furthermore, the Gamma-Poisson mixture produces better topic coherence scores than the Dirichlet-multinomial mixture model, thus making it a viable option for the challenging task of topic modelling of short text.

Evaluation of web service clustering using Dirichlet Multinomial Mixture model based approach for Dimensionality Reduction in service representation

In recent years, mainly the functionality of services are described in a short natural text language. Keyword-based searching for web service discovery is not efficient for providing relevant results. When services are clustered according to the similarity, then it reduces search space and due to that search time is also reduced in the web service discovery process. So in the domain of web service clustering, basically topic modeling techniques like Latent Dirichlet Allocation (LDA), Correlated Topic Model (CTM), Hierarchical Dirichlet Processing (HDP), etc. are adopted for dimensionality reduction and feature representation of services in vector space. But as the services are described in the form of short text, so these techniques are not efficient due to lack of occurring words, limited content, etc. In this paper, the performance of web service clustering is evaluated by applying various topic modeling techniques with different clustering algorithms on the crawled dataset from ProgrammableWeb repository. Gibbs Sampling algorithm for Dirichlet Multinomial Mixture (GSDMM) model is proposed as a dimensionality reduction and feature representation of services to overcome the limitations of short text clustering. Results show that GSDMM with K-Means or Agglomerative clustering is outperforming all other methods. The performance of clustering is evaluated based on three extrinsic and two intrinsic evaluation criteria. Dimensionality reduction achieved by GSDMM is 90.88%, 88.84%, and 93.13% on three real-time crawled datasets, which is satisfactory as the performance of clustering is also enhanced by deploying this technique.

Topic Modeling for Short Texts via Word Embedding and Document Correlation

Topic modeling is a widely studied foundational and interesting problem in the text mining domains. Conventional topic models based on word co-occurrences infer the hidden semantic structure from a corpus of documents. However, due to the limited length of short text, data sparsity impedes the inference process of conventional topic models and causes unsatisfactory results on short texts. In fact, each short text usually contains a limited number of topics, and understanding semantic content of short text needs to the relevant background knowledge. Inspired by the observed information, we propose a regularized non-negative matrix factorization topic model for short texts, named TRNMF. The proposed model leverages pre-trained distributional vector representation of words to overcome the data sparsity problem of short texts. Meanwhile, the method employs the clustering mechanism under document-to-topic distributions during the topic inference by using Gibbs Sampling Dirichlet Multinomial Mixture model. TRNMF integrates successfully both word co-occurrence regularization and sentence similarity regularization into topic modeling for short texts. Through extensive experiments on constructed real-world short text corpus, experimental results show that TRNMF can achieve better results than the state-of-the-art methods in term of topic coherence measure and text classification task.

Collaborative Online Edge Caching With Bayesian Clustering in Wireless Networks

In this article, we study the edge caching problem by considering the heterogeneous context with unknown users’ preferences. The cache provider (CP) can personalize the users’ storage based on available data to maximize the overall cache hit rate, accounting for the dynamic natures of both mobile edge cache scenarios and the users’ preferences. Toward this end, we introduce an online Bayesian clustering caching algorithm for the CP to autonomously learn the users’ interactive cache hit data in a collaborative way while maintaining sustainable scalability. Specifically, a Bayesian generative framework called the Dirichlet multinomial mixture (DMM) model is used to describe the uncertainty about the latent number of users’ clusters, each of which consists of the users with the same preference. Then, a dynamic clustering policy is proposed to obtain both the underlying mapping of users to clusters and the preferences of each cluster by using a collapsed Gibbs sampling algorithm. Subsequently, cache decisions are made according to the generated mappings by extending the traditional cache bandit algorithm to a new bandit mechanism with clusters of arms, capable of expediting the learning process between the exploitation and exploration. We theoretically characterize the value of dynamic Bayesian clustering for the long-term edge caching scenario with respect to the regret incurred by the noncluster schemes. Finally, using a real-world data set, our numerical results show that the proposed scheme outperforms the caching algorithms without clustering in the uncertain network scenario.