Multinomial Dirichlet Distribution Research Articles

Clustering multivariate count data via Dirichlet-multinomial network fusion

Clustering of multivariate count data has widespread applications in areas such as text analysis and microbiome studies. The need to account for overdispersion generally results in a nonconvex loss function, which does not fit into the existing convex clustering framework. Moreover, prior knowledge of a network over the samples, often available from citation or similarity relationships, is not taken into account. We introduce Dirichlet-multinomial network fusion (DMNet) for clustering multivariate count data, which models the samples via Dirichlet-multinomial distributions with individual parameters and employs a weighted group L1 fusion penalty to pursue homogeneity over a prespecified network. To circumvent the nonconvexity issue, we present two exponential family approximations to the Dirichlet-multinomial distribution, which are amenable to efficient optimization and theoretical analysis. We derive an ADMM algorithm and establish nonasymptotic error bounds for the proposed methods. Our bounds involve a trade-off between the connectivity of the network and its fidelity to the true parameter. The usefulness of our methods is illustrated through simulation studies and two text clustering applications.

EDClust: an EM-MM hybrid method for cell clustering in multiple-subject single-cell RNA sequencing.

Single-cell RNA sequencing (scRNA-seq) has revolutionized biological research by enabling the measurement of transcriptomic profiles at the single-cell level. With the increasing application of scRNA-seq in larger-scale studies, the problem of appropriately clustering cells emerges when the scRNA-seq data are from multiple subjects. One challenge is the subject-specific variation; systematic heterogeneity from multiple subjects may have a significant impact on clustering accuracy. Existing methods seeking to address such effects suffer from several limitations. We develop a novel statistical method, EDClust, for multi-subject scRNA-seq cell clustering. EDClust models the sequence read counts by a mixture of Dirichlet-multinomial distributions and explicitly accounts for cell-type heterogeneity, subject heterogeneity and clustering uncertainty. An EM-MM hybrid algorithm is derived for maximizing the data likelihood and clustering the cells. We perform a series of simulation studies to evaluate the proposed method and demonstrate the outstanding performance of EDClust. Comprehensive benchmarking on four real scRNA-seq datasets with various tissue types and species demonstrates the substantial accuracy improvement of EDClust compared to existing methods. The R package is freely available at https://github.com/weix21/EDClust. Supplementary data are available at Bioinformatics online.

Studying the Third Cumulant of the Mixture of Dirichlet-multinomial Distributions

Traditionally, the overdispersion parameter ϕ is estimated by using Pearson’s lack of fit statistic X2or the Deviance statistic D, which do not perform well in the case of sparse data. This paper particularly focuses on an estimator ϕnew of overdispersion parameter which was proposed for sparse multinomial data. The estimator was derived on the basis of an assumption on the 3rd cumulant of the response variable.When the data comes from the Dirichlet-multinomial distribution ϕnew is known to have the lowest root mean squared error comparing to the other three estimators. In this paper the 1st to 3rd order raw moments of the finite mixture of Dirichlet-multinomial distributions are derived, which results in complicated mathematical expressions. Furthermore, it is found that the 3rd cumulant of this mixture does not satisfy the assumption which is considered in the derivation of ϕnew . Dhaka Univ. J. Sci. 69(2): 96-100, 2021 (July)

Heterogeneous Growth Cycles

This paper focuses on the Dirichlet and the Dirichlet-multinomial distributions to analyze one definition of the cycle based on three economic performance regimens: augmentation, stagnation and diminution. We use data from 142 economies grouped in seven clusters during the second half of the last century, a period which has witnessed various structural changes. Furthermore, this paper advances to investigate what the probabilities are of reaching significant positive growth rates ⎯the frequency of the augmentation regimen⎯ which prevailed during the decade from 1950 to 1960, using Bayesian statistics.

Mixture‐based clustering for count data using approximated Fisher Scoring and Minorization–Maximization approaches

AbstractThe multinomial distribution has been widely used to model count data. To increase clustering efficiency, we use an approximation to the Fisher scoring algorithm, which is more robust regarding the choice of initial parameter values. Then, we use a novel approach to estimate the optimal number of components, based on minimum message length criterion. Moreover, we consider a generalization of the multinomial model obtained by introducing the Dirichlet as prior, yielding the Dirichlet Compound Multinomial (DCM). Even though DCM can address the burstiness phenomenon of count data, the presence of Gamma function in its density function usually leads to undesired complications. In this article, we use two alternative representations of DCM distribution to perform clustering based on finite mixture models, where the mixture parameters are estimated using the minorization–maximization framework. To evaluate and compare the performance of our proposed models, we have considered three challenging real‐world applications that involve high‐dimensional count vectors, namely, sentiment analysis, facial expression recognition, and human action recognition. The results show that the proposed algorithms increase the clustering efficiency of their respective models remarkably, and the best results are achieved by the second parametrization of DCM, which can accommodate over‐dispersed count data.

Efficacy and toxicity monitoring via Bayesian predictive probabilities in phase II clinical trials

Bayesian monitoring strategies based on predictive probabilities are widely used in phase II clinical trials that involve a single efficacy binary variable. The essential idea is to control the predictive probability that the trial will show a conclusive result at the scheduled end of the study, given the information at the interim stage and the prior beliefs. In this paper, we present an extension of this approach to incorporate toxicity considerations in single-arm phase II trials. We consider two binary endpoints representing response and toxicity of the experimental treatment and define the result as successful at the conclusion of the study if the posterior probability of an high efficacy and that of a small toxicity are both sufficiently large. At any interim look, the Multinomial-Dirichlet distribution provides the predictive probability of each possible combination of future efficacy and toxicity outcomes. It is exploited to obtain the predictive probability that the trial will yield a positive outcome, if it continues to the planned end. Different possible interim situations are considered to investigate the behaviour of the proposed predictive rules and the differences with the monitoring strategies based on posterior probabilities are highlighted. Simulation studies are also performed to evaluate the frequentist operating characteristics of the proposed design and to calibrate the design parameters.

Global Household Energy Model: A Multivariate Hierarchical Approach to Estimating Trends in The Use of Polluting and Clean Fuels for Cooking

SummaryIn 2017 an estimated 3 billion people used polluting fuels and technologies as their primary cooking solution, with 3.8 million deaths annually attributed to household exposure to the resulting fine particulate matter air pollution. Currently, health burdens are calculated by using aggregations of fuel types, e.g. solid fuels, as country level estimates of the use of specific fuel types, e.g. wood and charcoal, are unavailable. To expand the knowledge base about effects of household air pollution on health, we develop and implement a novel Bayesian hierarchical model, based on generalized Dirichlet–multinomial distributions, that jointly estimates non-linear trends in the use of eight key fuel types, overcoming several data-specific challenges including missing or combined fuel use values. We assess model fit by using within-sample predictive analysis and an out-of-sample prediction experiment to evaluate the model's forecasting performance.

Mixtures of Dirichlet-Multinomial distributions for supervised and unsupervised classification of short text data

Topic detection in short textual data is a challenging task due to its representation as high-dimensional and extremely sparse document-term matrix. In this paper we focus on the problem of classifying textual data on the base of their (unique) topic. For unsupervised classification, a popular approach called Mixture of Unigrams consists in considering a mixture of multinomial distributions over the word counts, each component corresponding to a different topic. The multinomial distribution can be easily extended by a Dirichlet prior to the compound mixtures of Dirichlet-Multinomial distributions, which is preferable for sparse data. We propose a gradient descent estimation method for fitting the model, and investigate supervised and unsupervised classification performance on real empirical problems.

A latent allocation model for the analysis of microbial composition and disease

BackgroundEstablishing the relationship between microbiota and specific diseases is important but requires appropriate statistical methodology. A specialized feature of microbiome count data is the presence of a large number of zeros, which makes it difficult to analyze in case-control studies. Most existing approaches either add a small number called a pseudo-count or use probability models such as the multinomial and Dirichlet-multinomial distributions to explain the excess zero counts, which may produce unnecessary biases and impose a correlation structure taht is unsuitable for microbiome data.ResultsThe purpose of this article is to develop a new probabilistic model, called BERnoulli and MUltinomial Distribution-based latent Allocation (BERMUDA), to address these problems. BERMUDA enables us to describe the differences in bacteria composition and a certain disease among samples. We also provide a simple and efficient learning procedure for the proposed model using an annealing EM algorithm.ConclusionWe illustrate the performance of the proposed method both through both the simulation and real data analysis. BERMUDA is implemented with R and is available from GitHub (https://github.com/abikoushi/Bermuda).

A Dirichlet-Multinomial Bayes Classifier for Disease Diagnosis with Microbial Compositions.

Dysbiosis of microbial communities is associated with various human diseases, raising the possibility of using microbial compositions as biomarkers for disease diagnosis. We have developed a Bayes classifier by modeling microbial compositions with Dirichlet-multinomial distributions, which are widely used to model multicategorical count data with extra variation. The parameters of the Dirichlet-multinomial distributions are estimated from training microbiome data sets based on maximum likelihood. The posterior probability of a microbiome sample belonging to a disease or healthy category is calculated based on Bayes' theorem, using the likelihood values computed from the estimated Dirichlet-multinomial distribution, as well as a prior probability estimated from the training microbiome data set or previously published information on disease prevalence. When tested on real-world microbiome data sets, our method, called DMBC (for Dirichlet-multinomial Bayes classifier), shows better classification accuracy than the only existing Bayesian microbiome classifier based on a Dirichlet-multinomial mixture model and the popular random forest method. The advantage of DMBC is its built-in automatic feature selection, capable of identifying a subset of microbial taxa with the best classification accuracy between different classes of samples based on cross-validation. This unique ability enables DMBC to maintain and even improve its accuracy at modeling species-level taxa. The R package for DMBC is freely available at https://github.com/qunfengdong/DMBC. IMPORTANCE By incorporating prior information on disease prevalence, Bayes classifiers have the potential to estimate disease probability better than other common machine-learning methods. Thus, it is important to develop Bayes classifiers specifically tailored for microbiome data. Our method shows higher classification accuracy than the only existing Bayesian classifier and the popular random forest method, and thus provides an alternative option for using microbial compositions for disease diagnosis.

Robust discriminative tracking via structured prior regularization

In this paper, we address the problem of tracking an object in a video sequence given its location in the first frame and no other information. Many existing discriminative tracking algorithms usually train a classifier in an on-line manner to separate the object of interest from the background. Slight inaccuracies in the tracking may result in incorrectly labelled training set, which can degrade the tracker. Although a number of approaches such as semi-supervised learning and multiple instance learning have been developed to address this problem, some critical issues still remain unsolved. This work aims to mitigate the shortcomings by exploiting a reliable generative model to support the discriminative learning process. A prior model based on a set of structured Dirichlet-multinomial distributions is proposed to preserve the target's structure information. This prior is then formulated as a regularization term in a training objective function, which casts the tracking task as a prior regularized semi-supervised learning problem. A multi-objective optimization method is developed to search for the solution, taking advantage of a decision maker inside to control the conflicts between different modules. The experiments show that this proposed method outperforms standard algorithms on challenging datasets. It is also demonstrated that the algorithm significantly mitigates the error accumulation effect.

Heinrich's pyramid and occupational safety: A statistical validation methodology

Heinrich’s pyramid, which postulates that the severity distribution of occupational accidents is relatively constant, is one of the main tools for management of occupational safety. However, existing evidence suggests that decreases in minor accidents have not always been followed by similar decreases in serious and fatal accidents. To test the statistical validity of the pyramid, we propose a Bayesian two-part model: the first part estimates accident rates using a Poisson-gamma distribution, and the second part estimates the proportion of minor, serious and fatal accidents using a Multinomial-Dirichlet distribution. If this proportion does not change when the accident rate is reduced, then the statistical validity of the pyramid is confirmed, but if it changes, then its validity is refuted. Our data cover more than 50,000 companies observed over 28months in Chile. Heinrich’s pyramid is confirmed to be statistically invalid for different economic activity sectors and geographic regions, but the discrepancy is so small that, for practical purposes, the pyramid is valid. We thus conclude that the occurrence of minor accidents is a useful signal for assessing and forecasting the overall safety performance of a firm.

Generative classification model for categorical data based on latent Gaussian process

In many machine learning applications such as computer-aided diagnosis, gene sequence analysis or natural language processing, categorical data appears. For small-scale data set with high dimensions, since relatively small proportion of possible categorical configurations are covered by training samples, conventional methods based on frequency information such as Dirichlet Compound Multinomial distribution usually runs into problems of over-fitting. Latent gaussian process is an effective bayesian non-parametric technique for categorical data modeling, which was proposed as an unsupervised method to embed unlabelled categorical data into a continuous and low-dimensional space through gaussian process. As a probabilistic generative model, latent gaussian process owns the ability of density estimation. In this paper, we propose a generative classification model as a supervised method for labelled categorical data, in which we use latent gaussian process to estimate the class-conditional densities. Since the complexity of gaussian process model can adapt to the size of training data, our method is able to effectively model small-sale categorical data. Experimental results show that our proposal can achieve better classification performance compared with other classification models for categorical data.

Estimating error models for whole genome sequencing using mixtures of Dirichlet-multinomial distributions.

MotivationAccurate identification of genotypes is an essential part of the analysis of genomic data, including in identification of sequence polymorphisms, linking mutations with disease and determining mutation rates. Biological and technical processes that adversely affect genotyping include copy-number-variation, paralogous sequences, library preparation, sequencing error and reference-mapping biases, among others.ResultsWe modeled the read depth for all data as a mixture of Dirichlet-multinomial distributions, resulting in significant improvements over previously used models. In most cases the best model was comprised of two distributions. The major-component distribution is similar to a binomial distribution with low error and low reference bias. The minor-component distribution is overdispersed with higher error and reference bias. We also found that sites fitting the minor component are enriched for copy number variants and low complexity regions, which can produce erroneous genotype calls. By removing sites that do not fit the major component, we can improve the accuracy of genotype calls.Availability and ImplementationMethods and data files are available at https://github.com/CartwrightLab/WuEtAl2017/ (doi:10.5281/zenodo.256858).Supplementary information Supplementary data is available at Bioinformatics online.

A Computational Bayesian Method for Generalized Semiparametric Regression Models

To better understand the power shift and the U.S. role compared to China and others regional actors, the Chicago Council on Global Affairs and the East Asia Institute (EAI) surveyed people in six countries - China, Japan, South Korea, Vietnam, Indonesian, and the United States - in the first half of 2008 about regional security and economic integration in Asia and about how these nations perceive each other (Bouton et al., 2010). There exists latent variance that cannot be adequately explained by parametric models. This is, in large part, due to the hidden structures and latent stories that from in unexpected ways. Therefore, a new Gibbs sampler is developed here in order to reveal preciously unseen structures and latent variances found in the survey dataset of Bouton et al. This new sampler is based upon the semiparametric regression, a well-known tool frequently utilized in order to capture the functional dependence between variables with fixed effect parametric and nonlinear regression. This is then extended to a generalized semiparametric regression for binary responses with logit and probit link function. The new sampler is then developed for the generalized linear mixed model with a nonparametric random effect. It is expressed as nonparametric regression with the multinomial-Dirichlet distribution for the number and positions of knots.

Analysis of composition of microbiomes: a novel method for studying microbial composition.

BackgroundUnderstanding the factors regulating our microbiota is important but requires appropriate statistical methodology. When comparing two or more populations most existing approaches either discount the underlying compositional structure in the microbiome data or use probability models such as the multinomial and Dirichlet-multinomial distributions, which may impose a correlation structure not suitable for microbiome data.ObjectiveTo develop a methodology that accounts for compositional constraints to reduce false discoveries in detecting differentially abundant taxa at an ecosystem level, while maintaining high statistical power.MethodsWe introduced a novel statistical framework called analysis of composition of microbiomes (ANCOM). ANCOM accounts for the underlying structure in the data and can be used for comparing the composition of microbiomes in two or more populations. ANCOM makes no distributional assumptions and can be implemented in a linear model framework to adjust for covariates as well as model longitudinal data. ANCOM also scales well to compare samples involving thousands of taxa.ResultsWe compared the performance of ANCOM to the standard t-test and a recently published methodology called Zero Inflated Gaussian (ZIG) methodology (1) for drawing inferences on the mean taxa abundance in two or more populations. ANCOM controlled the false discovery rate (FDR) at the desired nominal level while also improving power, whereas the t-test and ZIG had inflated FDRs, in some instances as high as 68% for the t-test and 60% for ZIG. We illustrate the performance of ANCOM using two publicly available microbial datasets in the human gut, demonstrating its general applicability to testing hypotheses about compositional differences in microbial communities.ConclusionAccounting for compositionality using log-ratio analysis results in significantly improved inference in microbiota survey data.

Worldwide F(ST) estimates relative to five continental-scale populations.

We estimate the population genetics parameter (also referred to as the fixation index) from short tandem repeat (STR) allele frequencies, comparing many worldwide human subpopulations at approximately the national level with continental-scale populations. is commonly used to measure population differentiation, and is important in forensic DNA analysis to account for remote shared ancestry between a suspect and an alternative source of the DNA. We estimate comparing subpopulations with a hypothetical ancestral population, which is the approach most widely used in population genetics, and also compare a subpopulation with a sampled reference population, which is more appropriate for forensic applications. Both estimation methods are likelihood-based, in which is related to the variance of the multinomial-Dirichlet distribution for allele counts. Overall, we find low values, with posterior 97.5 percentiles when comparing a subpopulation with the most appropriate population, and even for inter-population comparisons we find . These are much smaller than single nucleotide polymorphism-based inter-continental estimates, and are also about half the magnitude of STR-based estimates from population genetics surveys that focus on distinct ethnic groups rather than a general population. Our findings support the use of up to 3% in forensic calculations, which corresponds to some current practice.

Next‐generation sequencing for molecular ecology: a caveat regarding pooled samples

We develop a model based on the Dirichlet-compound multinomial distribution (CMD) and Ewens sampling formula to predict the fraction of SNP loci that will appear fixed for alternate alleles between two pooled samples drawn from the same underlying population. We apply this model to next-generation sequencing (NGS) data from Baltic Sea herring recently published by (Corander et al., 2013, Molecular Ecology, 2931-2940), and show that there are many more fixed loci than expected in the absence of genetic structure. However, we show through coalescent simulations that the degree of population structure required to explain the fraction of alternatively fixed SNPs is extraordinarily high and that the surplus of fixed loci is more likely a consequence of limited representation of individual gene copies in the pooled samples, than it is of population structure. Our analysis signals that the use of NGS on pooled samples to identify divergent SNPs warrants caution. With pooled samples, it is hard to diagnose when an NGS experiment has gone awry; especially when NGS data on pooled samples are of low read depth with a limited number of individuals, it may be worthwhile to temper claims of unexpected population differentiation from pooled samples, pending verification with more reliable methods or stricter adherence to recommended sampling designs for pooled sequencing e.g. Futschik & Schlötterer 2010, Genetics, 186, 207; Gautier et al., 2013a, Molecular Ecology, 3766-3779). Analysis of the data and diagnosis of problems is easier and more reliable (and can be less costly) with individually barcoded samples. Consequently, for some scenarios, individual barcoding may be preferable to pooling of samples.

A random effects population dynamics model based on proportions-at-age and removal data for estimating total mortality

Catch curves are widely used to estimate total mortality for exploited marine populations. The usual population dynamics model assumes constant recruitment across years and constant total mortality. We extend this to include annual recruitment and annual total mortality. Recruitment is treated as an uncorrelated random effect, while total mortality is modelled by a random walk. Data requirements are minimal as only proportions-at-age and total catches are needed. We obtain the effective sample size for aggregated proportion-at-age data based on fitting Dirichlet-multinomial distributions to the raw sampling data. Parameter estimation is carried out by approximate likelihood. We use simulations to study parameter estimability and estimation bias of four model versions, including models treating mortality as fixed effects and misspecified models. All model versions were, in general, estimable, though for certain parameter values or replicate runs they were not. Relative estimation bias of final year total mortalities and depletion rates were lower for the proposed random effects model compared with the fixed effects version for total mortality. The model is demonstrated for the case of blue ling (Molva dypterygia) to the west of the British Isles for the period 1988 to 2011.

Modelling the evolutionary dynamics of viruses within their hosts: a case study using high-throughput sequencing.

Uncovering how natural selection and genetic drift shape the evolutionary dynamics of virus populations within their hosts can pave the way to a better understanding of virus emergence. Mathematical models already play a leading role in these studies and are intended to predict future emergences. Here, using high-throughput sequencing, we analyzed the within-host population dynamics of four Potato virus Y (PVY) variants differing at most by two substitutions involved in pathogenicity properties. Model selection procedures were used to compare experimental results to six hypotheses regarding competitiveness and intensity of genetic drift experienced by viruses during host plant colonization. Results indicated that the frequencies of variants were well described using Lotka-Volterra models where the competition coefficients βij exerted by variant j on variant i are equal to their fitness ratio, rj/ri. Statistical inference allowed the estimation of the effect of each mutation on fitness, revealing slight (s = −0.45%) and high (s = −13.2%) fitness costs and a negative epistasis between them. Results also indicated that only 1 to 4 infectious units initiated the population of one apical leaf. The between-host variances of the variant frequencies were described using Dirichlet-multinomial distributions whose scale parameters, closely related to the fixation index F ST, were shown to vary with time. The genetic differentiation of virus populations among plants increased from 0 to 10 days post-inoculation and then decreased until 35 days. Overall, this study showed that mathematical models can accurately describe both selection and genetic drift processes shaping the evolutionary dynamics of viruses within their hosts.