Dirichlet Priors Research Articles

A Bayesian cluster validity index

Selecting the appropriate number of clusters is a critical step in applying clustering algorithms. To assist in this process, various cluster validity indices (CVIs) have been developed. These indices are designed to identify the optimal number of clusters within a dataset. However, users may not always seek the absolute optimal number of clusters but rather a secondary option that better aligns with their specific applications. This realization has led us to introduce a Bayesian cluster validity index (BCVI), which builds upon existing indices. The BCVI utilizes either Dirichlet or generalized Dirichlet priors, resulting in the same posterior distribution. The proposed BCVI is evaluated using the Calinski-Harabasz, CVNN, Davies–Bouldin, silhouette, Starczewski, and Wiroonsri indices for hard clustering and the KWON2, Wiroonsri–Preedasawakul, and Xie–Beni indices for soft clustering as underlying indices. The performance of the proposed BCVI with that of the original underlying indices has been compared. The BCVI offers clear advantages in situations where user expertise is valuable, allowing users to specify their desired range for the final number of clusters. To illustrate this, experiments classified into three different scenarios are conducted. Additionally, the practical applicability of the proposed approach through real-world datasets, such as MRI brain tumor images are presented. These tools are published as a recent R package ‘BayesCVI’.

A data-driven analysis to determine the optimal number of topics 'K' for latent Dirichlet allocation model

Topic modeling is an unsupervised machine learning technique successfully used to classify and retrieve textual data. However, the performance of topic models is sensitive to selecting optimal hyperparameters, the number of topics 'K' and Dirichlet priors 'α' and 'β.' This data-driven analysis aims to determine the optimum number of topics, 'K,' within the latent Dirichlet allocation (LDA) model. This work utilizes three datasets, namely 20-Newsgroups news articles, Wikipedia articles, and Web of Science containing science articles, to assess and compare various 'K' values through the grid search approach. The grid search approach finds the best combination of hyperparameter values by trying all possible combinations to see which performs best. This research seeks to identify the 'K' that optimizes topic relevance, coherence, and model performance by leveraging statistical metrics, such as coherence scores, perplexity, and topic distribution quality. Through empirical analysis and rigorous evaluation, this work provides valuable insights for determining the ideal 'K' for LDA models.

Towards Evidential and Class Separable Open Set Object Detection

Detecting in open-world scenarios poses a formidable challenge for models intended for real-world deployment. The advanced closed set object detectors achieve impressive performance under the closed set setting, but often produce overconfident misprediction on unknown objects due to the lack of supervision. In this paper, we propose a novel Evidential Object Detector (EOD) to formulate the Open Set Object Detection (OSOD) problem from the perspective of Evidential Deep Learning (EDL) theory, which quantifies classification uncertainty by placing the Dirichlet Prior over the categorical distribution parameters. The task-specific customized evidential framework, equipped with meticulously designed model architecture and loss function, effectively bridges the gap between EDL theory and detection tasks. Moreover, we utilize contrastive learning as an implicit means of evidential regularization and to encourage the class separation in the latent space. Alongside, we innovatively model the background uncertainty to further improve the unknown discovery ability. Extensive experiments on benchmark datasets demonstrate the outperformance of the proposed method over existing ones.

Bayesian estimation of the Kullback-Leibler divergence for categorical systems using mixtures of Dirichlet priors.

In many applications in biology, engineering, and economics, identifying similarities and differences between distributions of data from complex processes requires comparing finite categorical samples of discrete counts. Statistical divergences quantify the difference between two distributions. However, their estimation is very difficult and empirical methods often fail, especially when the samples are small. We develop a Bayesian estimator of the Kullback-Leibler divergence between two probability distributions that makes use of a mixture of Dirichlet priors on the distributions being compared. We study the properties of the estimator on two examples: probabilities drawn from Dirichlet distributions and random strings of letters drawn from Markov chains. We extend the approach to the squared Hellinger divergence. Both estimators outperform other estimation techniques, with better results for data with a large number of categories and for higher values of divergences.

Parallel inference for cross-collection latent generalized Dirichlet allocation model and applications

Existing cross-collection topic models with document-topic representation encounter performance bottlenecks in large-scale datasets due to their reliance on Dirichlet priors and conventional inference schemes. These constraints become noticeable in models derived from the Latent Dirichlet Allocation (LDA) framework. To address these challenges, this paper introduces the GPU-accelerated cross-collection latent generalized Dirichlet allocation (gccLGDA) model. This innovative approach integrates the benefits of generalized Dirichlet (GD) distribution with the computational prowess of GPU-based parallel inference, offering enhanced cross-collection topic modeling. The gccLGDA employs the GD distribution presenting a more flexible prior with a comprehensive covariance structure, enabling a more nuanced capture of relationships between latent topics across different collections. Leveraging GPU for parallel inference, our model promises scalable and efficient training for expansive datasets, making it apt for large-scale data challenges. Through empirical evaluations in comparative text mining and document classification, we demonstrate the enhanced performance of the gccLGDA, highlighting its advantages over existing cross-collection topic models.

On the Performance of Dirichlet Prior Mixture of Generalized Linear Mixed Models for Zero Truncated Count Data

On the Performance of Dirichlet Prior Mixture of Generalized Linear Mixed Models for Zero Truncated Count Data

Online short text clustering using infinite extensions of discrete mixture models

AbstractShort text clustering is one of the fundamental tasks in natural language processing. Different from traditional documents, short texts are ambiguous and sparse due to their short form and the lack of recurrence in word usage from one text to another, making it very challenging to apply conventional machine learning algorithms directly. In this article, we propose two novel approaches for short texts clustering: collapsed Gibbs sampling infinite generalized Dirichlet multinomial mixture model infinite GSGDMM) and collapsed Gibbs sampling infinite Beta‐Liouville multinomial mixture model (infinite GSBLMM). We adopt two flexible and practical priors to the multinomial distribution where in the first one the generalized Dirichlet distribution is integrated, while the second one is based on the Beta‐Liouville distribution. We evaluate the proposed approaches on two famous benchmark datasets, namely, Google News and Tweet. The experimental results demonstrate the effectiveness of our models compared to basic approaches that use Dirichlet priors. We further propose to improve the performance of our methods with an online clustering procedure. We also evaluate the performance of our methods for the outlier detection task, in which we achieve accurate results.

Improving Bayesian radiological profiling of waste drums using Dirichlet priors, Gaussian process priors, and hierarchical modeling

We present three methodological improvements of our recently proposed approach for Bayesian inference of the radionuclide inventory in radioactive waste drums, from radiological measurements. First we resort to the Dirichlet distribution for the prior distribution of the isotopic vector. The Dirichlet distribution possesses the attractive property that the elements of its vector samples sum up to 1. Second, we demonstrate that such Dirichlet priors can be incorporated within an hierarchical modeling of the prior uncertainty in the isotopic vector, when prior information about isotopic composition is available. Our used Bayesian hierarchical modeling framework makes use of this available information but also acknowledges its uncertainty by letting to a controlled extent the information content of the indirect measurement data (i.e., gamma and neutron counts) shape the actual prior distribution of the isotopic vector. Third, we propose to regularize the Bayesian inversion by using Gaussian process (GP) prior modeling when inferring 1D spatially-distributed mass or, equivalently, activity distributions. As of uncertainty in the efficiencies, we keep using the same stylized drum modeling approach as proposed in our previous work to account for the source distribution uncertainty across the vertical direction of the drum. A series of synthetic tests followed by application to a real waste drum show that combining hierarchical modeling of the prior isotopic composition uncertainty together with GP prior modeling of the vertical Pu profile across the drum works well. We also find that our GP prior can handles both cases with and without spatial correlation. Of course, our GP prior modeling framework only makes sense in the context of spatial inference. Furthermore, the computational times involved by our approach are on the order of a few hours, say about 2, to provide uncertainty estimates for all variables of interest in the considered inverse problem. This warrants further investigations to speed up the inference.

Inferring taxonomic placement from DNA barcoding aiding in discovery of new taxa

Abstract Predicting the taxonomic affiliation of DNA sequences collected from biological samples is a fundamental step in biodiversity assessment. This task is performed by leveraging existing databases containing reference DNA sequences endowed with a taxonomic identification. However, environmental sequences can be from organisms that are either unknown to science or for which there are no reference sequences available. Thus, taxonomic novelty of a sequence needs to be accounted for when doing classification. We propose Bayesian nonparametric taxonomic classifiers, BayesANT, which use species sampling model priors to allow unobserved taxa to be discovered at each taxonomic rank. Using a simple product multinomial likelihood with conjugate Dirichlet priors at the lowest rank, a highly flexible supervised algorithm is developed to provide a probabilistic prediction of the taxa placement of each sequence at each rank. As an illustration, we run our algorithm on a carefully annotated library of Finnish arthropods (FinBOL). To assess the ability of BayesANT to recognize novelty and to predict known taxonomic affiliations correctly, we test it on two training‐test splitting scenarios, each with a different proportion of taxa unobserved in training. We show how our algorithm attains accurate predictions and reliably quantifies classification uncertainty, especially when many sequences in the test set are affiliated to taxa unknown in training. By enabling taxonomic predictions for DNA barcodes to identify unseen branches, we believe BayesANT will be of broad utility as a tool for DNA metabarcoding within bioinformatics pipelines.

Bayesian Analysis of Masked Competing Risks Data Based on Proportional Subdistribution Hazards Model

Masked issues can emerge when dealing with competing risk data. Such issues are exemplified by the cause of a particular failure not being directly exhibited for all units to observe but only proven to be a subset of possible causes of failure. For assessing the impact of explanatory variables (covariates) on the cumulative incidence function (CIF), a process of Bayesian analysis is discussed in this paper. The symmetry assumption is not imposed on the masking probabilities and independent Dirichlet priors assigned to them. The Markov Chain Monte Carlo (MCMC) technique is utilized to implement the Bayesian analysis. The effectiveness of the developed model is tested via numerical studies, including simulated and real data sets.

Efficient Bayesian estimation of permutation entropy with Dirichlet priors

Estimation of permutation entropy (PE) using Bayesian statistical methods is presented for systems where the ordinal pattern sampling follows an independent, multinomial distribution. The desired PE posterior distribution is demonstrated to closely approximate a standard Beta distribution whose hyperparameters can be computed directly from ordinal pattern counts. It is further shown that this Bayesian approach is a generalization of previously-published frequentist methods. Because Bayesian posterior distributions can be estimated for very short time series, this method enables PE analysis on data sets currently not compatible with existing methods due to their limited size. To demonstrate the power and flexibility of this technique, the PE of a semiconductor laser with optical feedback (SLWOF) is shown to be time-variant by visualizing how the PE posterior distribution varies over sequences of small (1000 point) partitions of the output time-series.

UNIVERSAL RISK BUDGETING

I juxtapose Cover’s vaunted universal portfolio selection algorithm ([Cover, TM (1991). Universal portfolios. Mathematical Finance, 1, 1–29]) with the modern representation of a portfolio as a certain allocation of risk among the available assets, rather than a mere allocation of capital. Thus, I define a Universal Risk Budgeting scheme that weights each risk budget, instead of each capital budget, by its historical performance record, á la Cover. I prove that my scheme is mathematically equivalent to a novel type of [Cover, TM and E Ordentlich (1996). Universal portfolios with side information. IEEE Transactions on Information Theory, 42, 348–363] universal portfolio that uses a new family of prior densities that have hitherto not appeared in the literature on universal portfolio theory. I argue that my universal risk budget, so-defined, is a potentially more perspicuous and flexible type of universal portfolio; it allows the algorithmic trader to incorporate, with advantage, his prior knowledge or beliefs about the particular covariance structure of instantaneous asset returns. Say, if there is some dispersion in the volatilities of the available assets, then the uniform or Dirichlet priors that are standard in the literature will generate a dangerously lopsided prior distribution over the possible risk budgets. In the author’s opinion, the proposed “Garivaltis prior” makes for a nice improvement on Cover’s timeless expert system, that is properly agnostic and open to different risk budgets from the very get-go. Inspired by [Jamshidian, F (1992). Asymptotically optimal portfolios. Mathematical Finance, 2, 131–150], the universal risk budget is formulated as a new kind of exotic option in the continuous time Black–Scholes market, with all the pleasure, elegance, and convenience that entails.

Multinomial naïve Bayesian classifier with generalized Dirichlet priors for high-dimensional imbalanced data

The data-level approach can be used to balance the class distribution in a data set, while generating synthetic instances in a high-dimensional space is an awful task. Naïve Bayesian classifier with multinomial model, called multinomial naïve Bayesian classifier, is a popular classification algorithm for high-dimensional data because of its computational efficiency. However, this algorithm generally cannot have a better performance than the other algorithms on high-dimensional imbalanced data. Generalized Dirichlet distribution that is defined on unit simplex can be priors for multinomial naïve Bayesian classifier. This study proposes methods to find noninformative generalized Dirichlet priors for multinomial naïve Bayesian classifier so that its performance on high-dimensional imbalanced data can be largely improved. The methods are tested on seven high-dimensional imbalanced data sets to demonstrate that the multinomial naïve Bayesian classifier with generalized Dirichlet priors can significantly outperform not only the original multinomial naïve Bayesian classifier, but also random forest and Ripper algorithm.

Integrated hierarchical models to inform management of transitional habitat and the recovery of a habitat specialist

AbstractQuantifying the contribution of habitat dynamics relative to intrinsic population processes in regulating species persistence remains an ongoing challenge in ecological and applied conservation. Understanding these drivers and their relationship is essential for managing habitat‐dependent species, especially those that specialize in transitional habitats. Limitations in the ability of natural disturbance to mediate transitional habitat dynamics have resulted in a decline in early‐ and mid‐successional vegetation structure and prompted the need for aggressive habitat management to replace natural perturbations and increase habitat structural complexity. We describe a collaborative effort with a group of independent land managers to design an adaptive management program for restoring an imperiled ecosystem and recovering declining populations of an endemic habitat specialist. We developed a set of integrated, hierarchical models to estimate management‐mediated transition rates among vegetation classes in two dominant scrub communities and the species response (local colonization and extinction probabilities) as a function of habitat state. Models were fit using a long‐term data set of habitat and occupancy observations from 361 Florida scrub‐jay territories across two Florida counties. Occupancy model results correspond closely to previous approaches of estimating differential survival and reproductive success associated with habitat conditions, with highest colonization and lowest extinction rates estimated for those habitat states found to have the highest rates of survival and reproduction. In addition to offering an innovative approach for jointly modeling habitat and species population dynamics, the program we describe will also be of interest from a management perspective by providing guidance for developing collaborative, adaptive management frameworks from the ground up. We engaged land managers via workshops to specify objectives and desired state‐variable conditions, identify management alternatives, and elicit consensus opinions on model parameters. Treating expert opinions as pseudo‐observations to define Dirichlet priors allowed us to make use of existing management knowledge. Formal learning was then accumulated by updating transition probability estimates as management activities were implemented over the study period. We believe this adaptive management framework provides a useful approach for increasing our understanding of complex ecological relationships and hope that it will be adopted by others who have interest in informing management and conservation efforts.

MAP segmentation in Bayesian hidden Markov models: a case study

We consider the problem of estimating the maximum posterior probability (MAP) state sequence for a finite state and finite emission alphabet hidden Markov model (HMM) in the Bayesian setup, where both emission and transition matrices have Dirichlet priors. We study a training set consisting of thousands of protein alignment pairs. The training data is used to set the prior hyperparameters for Bayesian MAP segmentation. Since the Viterbi algorithm is not applicable any more, there is no simple procedure to find the MAP path, and several iterative algorithms are considered and compared. The main goal of the paper is to test the Bayesian setup against the frequentist one, where the parameters of HMM are estimated using the training data.

Co-Occurrence Estimation from Aggregated Data with Auxiliary Information

Complete co-occurrence data are unavailable in many applications, including purchase records and medical histories, because of their high cost or privacy protection. Even with such applications, aggregated data would be available, such as the number of purchasers for each item and the number of patients with each disease. We propose a method for estimating the co-occurrence of items from aggregated data with auxiliary information. For auxiliary information, we use item features that describe the characteristics of each item. Although many methods have been proposed for estimating the co-occurrence given aggregated data, no existing method can use auxiliary information. We also use records of a small number of users. With our proposed method, we introduce latent co-occurrence variables that represent the amount of co-occurrence for each pair of items. We model a probabilistic generative process of the latent co-occurrence variables by a multinomial distribution with Dirichlet priors. The parameters of the Dirichlet priors are parameterized with neural networks that take the auxiliary information as input, where neural networks are shared across different item pairs. The shared neural networks enable us to learn unknown relationships between auxiliary information and co-occurrence using the data of multiple items. The latent co-occurrence variables and the neural network parameters are estimated by maximizing the sum of the likelihood of the latent co-occurrence variables and the likelihood of the small records. We demonstrate the effectiveness of our proposed method using user-item rating datasets.

Intuitive Joint Priors for Variance Parameters

Variance parameters in additive models are typically assigned independent priors that do not account for model structure. We present a new framework for prior selection based on a hierarchical decomposition of the total variance along a tree structure to the individual model components. For each split in the tree, an analyst may be ignorant or have a sound intuition on how to attribute variance to the branches. In the former case a Dirichlet prior is appropriate to use, while in the latter case a penalised complexity (PC) prior provides robust shrinkage. A bottom-up combination of the conditional priors results in a proper joint prior. We suggest default values for the hyperparameters and offer intuitive statements for eliciting the hyperparameters based on expert knowledge. The prior framework is applicable for R packages for Bayesian inference such as INLA and RStan. Three simulation studies show that, in terms of the application-specific measures of interest, PC priors improve inference over Dirichlet priors when used to penalise different levels of complexity in splits. However, when expressing ignorance in a split, Dirichlet priors perform equally well and are preferred for their simplicity. We find that assigning current state-of-the-art default priors for each variance parameter individually is less transparent and does not perform better than using the proposed joint priors. We demonstrate practical use of the new framework by analysing spatial heterogeneity in neonatal mortality in Kenya in 2010–2014 based on complex survey data.

Statistical compression of protein sequences and inference of marginal probability landscapes over competing alignments using finite state models and Dirichlet priors.

The information criterion of minimum message length (MML) provides a powerful statistical framework for inductive reasoning from observed data. We apply MML to the problem of protein sequence comparison using finite state models with Dirichlet distributions. The resulting framework allows us to supersede the ad hoc cost functions commonly used in the field, by systematically addressing the problem of arbitrariness in alignment parameters, and the disconnect between substitution scores and gap costs. Furthermore, our framework enables the generation of marginal probability landscapes over all possible alignment hypotheses, with potential to facilitate the users to simultaneously rationalize and assess competing alignment relationships between protein sequences, beyond simply reporting a single (best) alignment. We demonstrate the performance of our program on benchmarks containing distantly related protein sequences.Availability and implementationThe open-source program supporting this work is available from: http://lcb.infotech.monash.edu.au/seqmmligner.Supplementary information Supplementary data are available at Bioinformatics online.

Learning Bayesian networks by constrained Bayesian estimation

Bayesian networks (BNs) have become increasingly popular in recent years due to their wide-ranging applications in modeling uncertain knowledge. An essential problem about discrete BNs is learning conditional probability table (CPT) parameters. If training data are sparse, purely data-driven methods often fail to learn accurate parameters. Then, expert judgments can be introduced to overcome this challenge. Parameter constraints deduced from expert judgments can cause parameter estimates to be consistent with domain knowledge. In addition, Dirichlet priors contain information that helps improve learning accuracy. This paper proposes a constrained Bayesian estimation approach to learn CPTs by incorporating constraints and Dirichlet priors. First, a posterior distribution of BN parameters is developed over a restricted parameter space based on training data and Dirichlet priors. Then, the expectation of the posterior distribution is taken as a parameter estimation. As it is difficult to directly compute the expectation for a continuous distribution with an irregular feasible domain, we apply the Monte Carlo method to approximate it. In the experiments on learning standard BNs, the proposed method outperforms competing methods. It suggests that the proposed method can facilitate solving real-world problems. Additionally, a case study of Wine data demonstrates that the proposed method achieves the highest classification accuracy.

Bayesian model selection for the study of Hardy-Weinberg proportions and homogeneity of gender allele frequencies.

Standard statistical tests for Hardy-Weinberg equilibrium assume the equality of allele frequencies in the sexes, whereas tests for the equality of allele frequencies in the sexes assume Hardy-Weinberg equilibrium. This produces a circularity in the testing of genetic variants, which has recently been resolved with new frequentist likelihood and exact procedures. In this paper, we tackle the same problem by posing it as a Bayesian model comparison problem. We formulate an exhaustive set of ten alternative scenarios for biallelic genetic variants. Using Dirichlet and Beta priors for genotype and allele frequencies, we derive marginal likelihoods for all scenarios, and select the most likely scenario using the posterior probabilities that each of these scenarios is the one in place. Different from the usual frequentist testing approach, the Bayesian approach allows one to compare any number of models, and not just two at a time, and the models compared do not have to be nested. We illustrate our Bayesian approach with genetic data from the 1,000 genomes project and through a simulation study.