Multinomial Process Research Articles

Bayesian mixture models (in)consistency for the number of clusters

AbstractBayesian nonparametric mixture models are common for modeling complex data. While these models are well‐suited for density estimation, recent results proved posterior inconsistency of the number of clusters when the true number of components is finite, for the Dirichlet process and Pitman–Yor process mixture models. We extend these results to additional Bayesian nonparametric priors such as Gibbs‐type processes and finite‐dimensional representations thereof. The latter include the Dirichlet multinomial process, the recently proposed Pitman–Yor, and normalized generalized gamma multinomial processes. We show that mixture models based on these processes are also inconsistent in the number of clusters and discuss possible solutions. Notably, we show that a postprocessing algorithm introduced for the Dirichlet process can be extended to more general models and provides a consistent method to estimate the number of components.

A spatio-temporal model of multi-marker antimalarial resistance.

The emergence and spread of drug-resistant Plasmodium falciparum parasites have hindered efforts to eliminate malaria. Monitoring the spread of drug resistance is vital, as drug resistance can lead to widespread treatment failure. We develop a Bayesian model to produce spatio-temporal maps that depict the spread of drug resistance, and apply our methods for the antimalarial sulfadoxine-pyrimethamine. We infer from genetic count data the prevalences over space and time of various malaria parasite haplotypes associated with drug resistance. Previous work has focused on inferring the prevalence of individual molecular markers. In reality, combinations of mutations at multiple markers confer varying degrees of drug resistance to the parasite, indicating that multiple markers should be modelled together. However, the reporting of genetic count data is often inconsistent as some studies report haplotype counts, whereas some studies report mutation counts of individual markers separately. In response, we introduce a latent multinomial Gaussian process model to handle partially reported spatio-temporal count data. As drug-resistant mutations are often used as a proxy for treatment efficacy, point estimates from our spatio-temporal maps can help inform antimalarial drug policies, whereas the uncertainties from our maps can help with optimizing sampling strategies for future monitoring of drug resistance.

Extrema of a multinomial assignment process

AbstractWe study the asymptotic behaviour of the expectation of the maxima and minima of a random assignment process generated by a large matrix with multinomial entries. A variety of results is obtained for different sparsity regimes.

Strong mixing properties of discrete-valued time series with exogenous covariates

We derive strong mixing conditions for many existing discrete-valued time series models that include exogenous covariates in the dynamic. Our main contribution is to study how a mixing condition on the covariate process transfers to a mixing condition for the response. Using a coupling method, we first derive mixing conditions for some Markov chains in random environments, which gives a first result for some autoregressive categorical processes with strictly exogenous regressors. Our result is then extended to some infinite memory categorical processes. In the second part of the paper, we study autoregressive models for which the covariates are sequentially exogenous. Using a general random mapping approach on finite sets, we get explicit mixing conditions that can be checked for many categorical time series found in the literature, including multinomial autoregressive processes, ordinal time series and dynamic multiple choice models. We also study some autoregressive count time series using a somewhat different contraction argument. Our contribution fills an important gap for such models, presented here under a more general form, since such a strong mixing condition is often assumed in some recent works but no general approach is available to check it.

A Closed-Form EVSI Expression for a Multinomial Data-Generating Process

This paper derives analytic expressions for the expected value of sample information (EVSI), the expected value of distribution information, and the optimal sample size when data consists of independent draws from a bounded sequence of integers. Because of the challenges of creating tractable EVSI expressions, most existing work valuing data does so in one of three ways: (1) analytically through closed-form expressions on the upper bound of the value of data, (2) calculating the expected value of data using numerical comparisons of decisions made using simulated data to optimal decisions for which the underlying data distribution is known, or (3) using variance reduction as proxy for the uncertainty reduction that accompanies more data. For the very flexible case of modeling integer-valued observations using a multinomial data-generating process with Dirichlet prior, this paper develops expressions that (1) generalize existing beta-binomial computations, (2) do not require prior knowledge of some underlying “true” distribution, and (3) can be computed prior to the collection of any sample data.

A decision-making framework for selecting an MBSE language–A case study to ship pilotage

The increasing system complexity due to technological advances in recent decades requires the implementation of Model-Based System Engineering (MBSE) languages. While MBSE languages can support in creating detailed and precise system descriptions, requirements, functions, and component interactions, the development of several modelling languages has formed a challenge for system engineers to select a suitable modelling language for complex operations. Since the implementation of MBSE modelling languages is extensive and requires high resources, system engineers need to select the most suitable one for their project. Furthermore, the prepared system models are utilized by different end-users such as system engineers, operators, and marketing professionals. As a result, it is necessary to integrate the perception of end-users in the modelling language selection process. Hence, a decision-making support framework needs to be developed, which will incorporate the opinions of end-users in the selection process. Correspondingly, this study proposes a Multi-Criteria Decision-Making framework aimed to select the most appropriate and practical modelling language. The framework integrates the end-user’s perception in the selection process using the Technology Acceptance Model and reckons the MBSE language features as comparison criteria. To analyze the data collected from the end-users, integration of Multinomial Process Tree modelling and Bayesian inference is developed. The applicability of the proposed model was tested in a ship pilotage operation case study. The results show that the framework can support system engineers during the initial selection process of the MBSE modelling language.

A class of dependent Dirichlet processes via latent multinomial processes

We describe a procedure to introduce general dependence structures on a set of Dirichlet processes. Dependence can be in one direction to define a time series or in two directions to define spatial dependencies. More directions can also be considered. Dependence is induced via a set of latent processes and exploit the conjugacy property between the Dirichlet and the multinomial processes to ensure that the marginal law for each element of the set is a Dirichlet process. Dependence is characterized through the correlation between any two elements. Posterior distributions are obtained when we use the set of Dirichlet processes as prior distributions in a Bayesian nonparametric context. Posterior predictive distributions induce partially exchangeable sequences defined by generalized Pólya urns. A numerical example to illustrate is also included.

A multinomial process tree for reliability assessment of machinery in autonomous ships

Maritime Autonomous Surface Ships have received a significant amount of attention in recent projects. They promise a reduction in marine accidents and mitigation of human errors. Most of the ongoing research effort is directed toward autonomous navigation and cybersecurity. However, the importance of a machinery plant in the engine room that can operate reliably without human attendance is hardly investigated. To prevent failures in such systems and extend the interval between required human interventions, it is essential to improve their reliability. This paper aims to present a systematic approach to evaluate the reliability of an autonomous system under the influence of uncertain disruptions and to predict failure rates of unattended machinery plants. A Multinomial Process Tree is used to model failures in the main failure-sensitive components. Hierarchical Bayesian Inference is adopted to facilitate the prediction of frequencies of disruptive events and estimate the entire system's failure rate. The outcome of this research enables design strategies to improve the reliability of autonomous ships and prevent Fatal Technical Failure during the operation. This allows assessing whether a given machinery plant is sufficiently reliable to be used on unmanned ships. A case study is considered to demonstrate the application of the presented method.

CAN Algorithm: An Individual Level Approach to Identify Consequence and Norm Sensitivities and Overall Action/Inaction Preferences in Moral Decision-Making.

Recently, a multinomial process tree model was developed to measure an agent’s consequence sensitivity, norm sensitivity, and generalized inaction/action preferences when making moral decisions (CNI model). However, the CNI model presupposed that an agent considers consequences—norms—generalized inaction/action preferences sequentially, which is untenable based on recent evidence. Besides, the CNI model generates parameters at the group level based on binary categorical data. Hence, the C/N/I parameters cannot be used for correlation analyses or other conventional research designs. To solve these limitations, we developed the CAN algorithm to compute norm and consequence sensitivities and overall action/inaction preferences algebraically in a parallel manner. We re-analyzed the raw data of the original CNI model to test the methodological predictions. Our results demonstrate that: (1) the C parameter is approximately equal between the CNI model and CAN algorithm; (2) the N parameter under the CNI model approximately equals N/(1 − C) under the CAN algorithm; (3) the I parameter and A parameter are reversed around 0.5 – the larger the I parameter, the more the generalized inaction versus action preference and the larger the A parameter, the more overall action versus inaction preference; (4) tests of differences in parameters between groups with the CNI model and CAN algorithm led to almost the same statistical conclusion; (5) parameters from the CAN algorithm can be used for correlational analyses and multiple comparisons, and this is an advantage over the parameters from the CNI model. The theoretical and methodological implications of our study were also discussed.

Monitoring multinomial processes based on a weighted chi-square control chart

Abstract: Interpreting an out-of-control signal is a crucial step in monitoring categorical processes. For the Chi-Square Control Chart (CSCC), an out-of control situation does not specify if it was a process deterioration or a process improvement. For this reason, a weighted chi-square statistical control chart WSCC is proposed with different weighting categories in order to enable an accelerated disclosure of a control situation after a shift due to a deterioration of quality and on the other hand, decelerate an out of control situation after a shift due to a quality improvement. Furthermore, in comparison with Marcucci’s method, the new procedure provides an accurate and easier way to interpret several signals. In other words, the WSCC allows a faster detection of an out-of control situation in the case of a quality deterioration, however, an out-of control situation is not quickly detected in the case of a quality improvement. Indeed, comparative studies have been performed to find the best control chart for each combination. Concluding remarks with comments and recommendations are given based on Average Run Length (ARL) and standard deviation run length (SDRL).

ERRATUM: Linear combination of chi-squares for multinomial process monitoring

ERRATUM: Linear combination of chi-squares for multinomial process monitoring

Linear combination of chi-squares for multinomial process monitoring

Abstract: Marcucci (1985) proposed a chi square goodness of fit statistic based generalized p-chart for multinomial process monitoring. A chi square distribution quantile was considered as a control chart limit. A weighted chi square goodness of fit statistic-based control chart is proposed for multinomial process monitoring in this paper, where more important weights are advocated to poor quality categories. The statistic distribution is approximated by a well-known linear combination of chi squares distribution. The approximation is assessed through a simulation, an extreme percentile of the approximated distribution is used as an upper control chart limit and a comparison is carried out with a chi square goodness of fit statistic-based control chart. The average run length is used as a benchmark and the comparison is performed using simulations considering two process shifts scenarios. Under some restrictions, the weighted statistic-based control chart allows an earlier detection of process shift in case of deterioration and postpones out of control signals in case of improvement. This benefit is clearer when the process is improved by a decrease in the poor quality probability category and an increase in the best quality category probability.

ERRATUM: Monitoring multinomial processes based on a weighted chi-square control chart

ERRATUM: Monitoring multinomial processes based on a weighted chi-square control chart

Mapping global trends in vaccine confidence and investigating barriers to vaccine uptake: a large-scale retrospective temporal modelling study

SummaryBackgroundThere is growing evidence of vaccine delays or refusals due to a lack of trust in the importance, safety, or effectiveness of vaccines, alongside persisting access issues. Although immunisation coverage is reported administratively across the world, no similarly robust monitoring system exists for vaccine confidence. In this study, vaccine confidence was mapped across 149 countries between 2015 and 2019.MethodsIn this large-scale retrospective data-driven analysis, we examined global trends in vaccine confidence using data from 290 surveys done between September, 2015, and December, 2019, across 149 countries, and including 284 381 individuals. We used a Bayesian multinomial logit Gaussian process model to produce estimates of public perceptions towards the safety, importance, and effectiveness of vaccines. Associations between vaccine uptake and a large range of putative drivers of uptake, including vaccine confidence, socioeconomic status, and sources of trust, were determined using univariate Bayesian logistic regressions. Gibbs sampling was used for Bayesian model inference, with 95% Bayesian highest posterior density intervals used to capture uncertainty.FindingsBetween November, 2015, and December, 2019, we estimate that confidence in the importance, safety, and effectiveness of vaccines fell in Afghanistan, Indonesia, Pakistan, the Philippines, and South Korea. We found significant increases in respondents strongly disagreeing that vaccines are safe between 2015 and 2019 in six countries: Afghanistan, Azerbaijan, Indonesia, Nigeria, Pakistan, and Serbia. We find signs that confidence has improved between 2018 and 2019 in some EU member states, including Finland, France, Ireland, and Italy, with recent losses detected in Poland. Confidence in the importance of vaccines (rather than in their safety or effectiveness) had the strongest univariate association with vaccine uptake compared with other determinants considered. When a link was found between individuals' religious beliefs and uptake, findings indicated that minority religious groups tended to have lower probabilities of uptake.InterpretationTo our knowledge, this is the largest study of global vaccine confidence to date, allowing for cross-country comparisons and changes over time. Our findings highlight the importance of regular monitoring to detect emerging trends to prompt interventions to build and sustain vaccine confidence.FundingEuropean Commission, Wellcome, and Engineering and Physical Sciences Research Council.

The Pitman–Yor multinomial process for mixture modelling

SummaryDiscrete nonparametric priors play a central role in a variety of Bayesian procedures, most notably when used to model latent features, such as in clustering, mixtures and curve fitting. They are effective and well-developed tools, though their infinite dimensionality is unsuited to some applications. If one restricts to a finite-dimensional simplex, very little is known beyond the traditional Dirichlet multinomial process, which is mainly motivated by conjugacy. This paper introduces an alternative based on the Pitman–Yor process, which provides greater flexibility while preserving analytical tractability. Urn schemes and posterior characterizations are obtained in closed form, leading to exact sampling methods. In addition, the proposed approach can be used to accurately approximate the infinite-dimensional Pitman–Yor process, yielding improvements over existing truncation-based approaches. An application to convex mixture regression for quantitative risk assessment illustrates the theoretical results and compares our approach with existing methods.

A Process Dissociation Model of Implicit Rapid Revision in Response to Diagnostic Revelations.

Previous research has demonstrated that implicit evaluations can be reversed with exposure to a single impression-inconsistent behavior. But what exactly is changing when perceivers encounter diagnostic revelations about someone? One possibility is that rapid changes are occurring in the extent to which perceivers view the person positively or negatively. Another possibility is that they override the expression of initial evaluations through control-oriented processes. We conducted three studies (one preregistered) that used multinomial process trees to distinguish between these possibilities. We find consistent support across two different implicit measures that diagnostic behaviors result in rapid changes in evaluative processes. We obtained only inconsistent evidence for effects on more control-oriented processes. These findings thus help to reveal the cognitive processes underlying rapid implicit revision. Implications for theoretical perspectives on implicit attitudes are discussed.

GDTM: Graph-based Dynamic Topic Models

Dynamic Topic Modeling (DTM) is the ultimate solution for extracting topics from short texts generated in Online Social Networks (OSNs) like Twitter. It requires to be scalable and to be able to account for sparsity and dynamicity of short texts. Current solutions combine probabilistic mixture models like Dirichlet Multinomial or Pitman-Yor Process with approximate inference approaches like Gibbs Sampling and Stochastic Variational Inference to, respectively, account for dynamicity and scalability of DTM. However, these methods basically rely on weak probabilistic language models, which do not account for sparsity in short texts. In addition, their inference is based on iterative optimizations, which have scalability issues when it comes to DTM. We present GDTM, a single-pass graph-based DTM algorithm, to solve the problem. GDTM combines a context-rich and incremental feature representation method with graph partitioning to address scalability and dynamicity and uses a rich language model to account for sparsity. We run multiple experiments over a large-scale Twitter dataset to analyze the accuracy and scalability of GDTM and compare the results with four state-of-the-art models. In result, GDTM outperforms the best model by 11% on accuracy and performs by an order of magnitude faster while creating four times better topic quality over standard evaluation metrics.

A model with space-varying regression coefficients for clustering multivariate spatial count data.

Multivariate spatial count data are often segmented by unobserved space-varying factors that vary across space. In this setting, regression models that assume space-constant covariate effects could be too restrictive. Motivated by the analysis of cause-specific mortality data, we propose to estimate space-varying effects by exploiting a multivariate hidden Markov field. It models the data by a battery of Poisson regressions with spatially correlated regression coefficients, which are driven by an unobserved spatial multinomial process. It parsimoniously describes multivariate count data by means of a finite number of latent classes. Parameter estimation is carried out by composite likelihood methods, that we specifically develop for the proposed model. In a case study of cause-specific mortality data in Italy, the model was capable to capture the spatial variation of gender differences and ageeffects.

Delay analysis for aggregate traffic based on martingales theory

Based on martingale theory, the authors explore a novel method to analyse the bound of delay violation probability for aggregate traffic in this study. The proposed arrival model, four-state Markov modulated multinomial process, describes the behaviour of mixed service flows, which depicts the packet arrival process of two interweaved services and highlights the arrival characteristics of the batch, bursty, and heterogeneous. The traffic of data service is modelled by the interrupted multinomial process to depict the randomness and discontinuity. Relying on martingale theory, arrival-supermartingales provide a model for integrating multiple heterogeneous services. They construct a supermartingale process of the backlog in the buffer, which enables the analysis of delay for the queuing system with multiple arrivals and a single server. Finally, the delay of aggregate traffic is featured by the parameters of its corresponding supermartingales. Based on the stopping time theory of martingales, the delay violation probability bound is derived, which matches the simulation results well and is tight.

How success enhances self-serving bias: A multinomial process model of the Implicit Association Test

Self-serving bias is individuals' belief that leads them to blame external forces when bad things happen and to give themselves credit when good things happen. To evaluate how underlying evaluative associations toward the self or others differ between individuals, and/or how the regulation mechanism of the influence of such associations differs, we used a multinomial process model to measure the underlying implicit self-esteem in these processes with 56 Chinese undergraduate students. The results indicated that participants assessed themselves as being better than others when their performance was followed by a desirable outcome. Subsequent application of the quadruple processes showed that both activation of positive associations toward self and regulation of the associations played important roles in attitudinal responses. Our findings may provide a supplementary explanation to that of previous results, promoting understanding of the mechanism underlying self-serving bias.