Dirichlet process multi-state mixture models

Journal Article Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models Get access H Ishwaran, H Ishwaran Search for other works by this author on: Oxford Academic Google Scholar M Zarepour M Zarepour Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 87, Issue 2, June 2000, Pages 371–390, https://doi.org/10.1093/biomet/87.2.371 Published: 01 June 2000

Customary modeling for continuous point-referenced data assumes a Gaussian process that is often taken to be stationary. When such models are fitted within a Bayesian framework, the unknown parameters of the process are assumed to be random, so a random Gaussian process results. Here we propose a novel spatial Dirichlet process mixture model to produce a random spatial process that is neither Gaussian nor stationary. We first develop a spatial Dirichlet process model for spatial data and discuss its properties. Because of familiar limitations associated with direct use of Dirichlet process models, we introduce mixing by convolving this process with a pure error process. We then examine properties of models created through such Dirichlet process mixing. In the Bayesian framework, we implement posterior inference using Gibbs sampling. Spatial prediction raises interesting questions, but these can be handled. Finally, we illustrate the approach using simulated data, as well as a dataset involving precipitation measurements over the Languedoc-Roussillon region in southern France.

Likelihood-free methods, like Approximate Bayesian Computation (ABC), have been extensively used in model-based statistical inference with intractable likelihood functions. When combined with Sequential Monte Carlo (SMC) algorithms they constitute a powerful approach for parameter estimation and model selection of mathematical models of complex biological systems. A crucial step in the ABC-SMC algorithms, significantly affecting their performance, is the propagation of a set of parameter vectors through a sequence of intermediate distributions using Markov kernels. In this article, we employ Dirichlet process mixtures (DPMs) to design optimal transition kernels and we present an ABC-SMC algorithm with DPM kernels. We illustrate the use of the proposed methodology using real data for the canonical Wnt signaling pathway. A multi-compartment model of the pathway is developed and it is compared to an existing model. The results indicate that DPMs are more efficient in the exploration of the parameter space and can significantly improve ABC-SMC performance. In comparison to alternative sampling schemes that are commonly used, the proposed approach can bring potential benefits in the estimation of complex multimodal distributions. The method is used to estimate the parameters and the initial state of two models of the Wnt pathway and it is shown that the multi-compartment model fits better the experimental data. Python scripts for the Dirichlet Process Gaussian Mixture model and the Gibbs sampler are available at https://sites.google.com/site/kkoutroumpas/software konstantinos.koutroumpas@ecp.fr.

ABSTRACTThe recovery of dead marked individuals, either alone or in combination with encounters of these individuals while alive, is an important source of data for estimating survival in birds, mammals, and fish. Various models have been developed to analyze such data in a Bayesian framework, including single‐state and multistate state‐space models, marginalized state‐space models, and multinomial models. An overview of the different formulations, together with an assessment of their parameter accuracy, computational efficiency, and flexibility in covariate modeling, is lacking so far. We assessed 13 models based on data simulation and analysis with the widely used R‐based software NIMBLE and JAGS. We found that all the models evaluated produced accurate parameter estimates, with the exception of the multistate state‐space models, which produced biased parameter estimates. This is because the standard MCMC samplers required for Bayesian inference do not work properly for this model. Although such multistate models work correctly in the frequentist framework, they should not be used in the Bayesian framework unless specially developed samplers are used. Instead, single‐state state‐space models, marginalized multistate state‐space models, multinomial multistate models, or reparameterized multistate models should be used. The marginalized state‐space and multinomial models were the most computationally efficient. The models evaluated do not differ in their ability to model temporal covariates but do differ for individual continuous covariates. The latter can be modeled in state‐space models but not in multinomial models. We also show that single‐state models can be formulated for the joint analysis of dead‐recovery and live encounter data, which are usually modeled with multistate models. This facilitates the inclusion of further auxiliary data and results in a computationally efficient model. We expect our overview to help ecologists decide which model to use when estimating survival from dead‐recovery data in the Bayesian framework.

The object of the study is the behavior of stock market volatility in response to sudden shocks and crisis-driven fluctuations, with a specific focus on capturing its complex temporal structure and memory effects. One of the biggest challenges in this domain lies in the inherent stochastic nature of volatility: it evolves irregularly over time, cannot be directly observed, and must be estimated from indirect indicators. Conventional models, particularly those grounded in classical Brownian motion, often fall short in accurately representing such dynamics, as they neglect the long-range dependence – or “market memory” – commonly observed in real financial time series. This oversight can lead to significant errors in volatility estimation, especially during phases of market turbulence such as financial crises or global events. A fractional diffusion framework was used during the study to model asset price dynamics, incorporating a time-dependent and initially unknown volatility function. This approach relies on fractional Brownian motion, whose non-Markovian properties enable the model to effectively account for long-term correlations in market behavior. To estimate the volatility, it is possible to employ statistical tools based on p-variations, which allowed to compute the Hurst index and reconstruct the underlying path of realized volatility with high sensitivity to structural market changes. It is possible to obtain that this method significantly improves the accuracy of volatility tracking, particularly under stress conditions, such as those observed during the 2020 COVID-19 crisis. It is connected to the fact that the suggested method has a number of features, in particular its ability to incorporate memory effects and to respond adaptively to high-frequency data variations. Thanks to that, let’s manage to capture abrupt volatility spikes and sustained market uncertainty more precisely. Compared to the standard models, it is possible to achieve the following advantages: enhanced responsiveness to market dynamics, improved reliability of volatility forecasts during crisis periods, and a more realistic reflection of financial market complexity.

The Dirichlet process plays a dominant role as a prior in Bayesian nonparametrics leading to the development of a wide variety of inferential procedures. In this chapter we give a comprehensive account of the Dirichlet process and its immediate variants—the Dirichlet invariant and mixture of Dirichlet processes. Starting with its formal definition and alternative representations which include the seminal Sethuraman representation, we proceed to discuss in depth many of its properties including the most important one, the conjugacy property, which makes posterior computations easy by simply updating its parameter. Its marginal distribution is characterized as the generalized Polya urn scheme, also known as the Chinese restaurant process, and is shown to provide a tool to sample the Dirichlet process easily. We identify a major limitation that with probability one, it selects a random discrete distribution. However, it is shown in the next chapter that the discreteness proves to be a useful feature in formulating a variety of Dirichlet mixture models. Next, we present the Dirichlet invariant and symmetrized Dirichlet prior processes which are found to be appropriate when the unknown distribution function is known a priori to be is structured such as being invariant under a finite group of transformations or being symmetric.To address the inadequacy of the Dirichlet process in dealing with certain types of data such as bioassay or right censored data, a mixture of Dirichlet processes is developed by treating the parameter of the Dirichlet process as random having a certain parametric distribution. This is described next along with its various properties. The mixtures of Dirichlet processes are shown to be extremely useful in developing various hierarchical and mixture models, some of which are presented in this chapter and others in the next chapter. A major hurdle in implementation of these models is that the posterior distributions needed to proceed with the Bayesian analysis are often found to be intractable making it necessary to generate them via simulation. Consequently, various computational procedures based on Monte Carlo Markov chain developed in the literature using Gibbs sampler, blocked Gibbs samples, slice and retrospective sampling are described here sparingly and relevant steps of algorithms proposed by the respective authors are included here for ease of understanding. Finally, other extensions of the Dirichlet process such as multinomial Dirichlet process, multivariate and generalized Dirichlet processes are briefly outlined.

Target detection has been found to enhance memory for concurrently presented stimuli under dual-task conditions. This "attentional boost effect" is reminiscent of findings in the event memory literature, where conditions giving rise to event boundaries have been shown to enhance memory for boundary items. Target detection commonly requires a working memory update (e.g., adding to a covert mental target count), which is also thought to be a key contributor to creating event boundaries. However, whether target detection impacts temporal memory in similar ways as event boundaries remains unknown, because these two parallel literatures have used different types of memory tests, making direct comparisons difficult. In a preregistered experiment with sequential Bayes factor design, we examined whether target detection influences temporal binding between items by inserting target and nontarget stimuli during encoding of trial-unique object images, andthen comparing subsequent temporal order and distance memory for image pairs that span a target or nontarget. We found that target detection enhanced recognition memory for target trial images but had no effect on temporal binding between items. In a follow-up experiment, we showed that when the encoding task required updating of task set rather than target count, event segmentation-related temporal memory effects were observed. These results document that target detection as such does not disrupt inter-item associations in memory, and that attention orienting in the absence of updating task sets does not create event boundaries. This suggests a key distinction between declarative and procedural working memory updates in segmenting events in memory.

We proposed an approach that has the ability to detect spatial clusters with skewed or irregular distributions. A mixture of Dirichlet processes (DP) was used to describe spatial distribution patterns. The effects of different batches of data collection efforts were also modeled with a Dirichlet process. To cluster spatial foci, a birth-death process was applied due to its advantage of easier jumping between different numbers of clusters. Inferences of parameters including clustering were drawn under a Bayesian framework. Simulations were used to demonstrate and assess the method. We applied the method to an fMRI meta-analysis dataset to identify clusters of foci corresponding to different emotions.

The problem of model-based state observer design for the chemostat microbial cultivation is considered. General chemostat model accounting for the memory (delay) effects is used for the state observer design. The memory effects influencing the process dynamics are described by zero-order exponential memory functions in the expressions of the specific growth rate and specific consumption rate. Two modifications of the general model (µ-type and S-type models) are used for the state observer design. In the first modification, the memory effects are accounted for by weighted averages of the specific growth and consumption rates' past values. The second modification assumes that the memory effects are accounted for by weighted averages of the past values of the substrate concentration only. Particular cases, when the memory effects are taken into account in the specific growth and consumption rates by equal memory functions and when the memory effects are taken into account in the expression of the specific growth rate only, are considered. The proposed procedure is based on the Kalman filtering method under the assumption for full knowledge of the process kinetics and on-line measurement of the limiting substrate concentration. As a case study, a model of the continuous growth of a strain Saccharomyces cerevisiae on a glucose-limited medium is considered. The robustness of the observer with respect to the adaptability parameter variations is studied.

Multistate models, that is, models with more than two distributions, are preferred over single-state probability models in modeling the distribution of travel time. Literature review indicated that the finite multistate modeling of travel time using lognormal distribution is superior to other probability functions. In this study, we extend the finite multistate lognormal model of estimating the travel time distribution to unbounded lognormal distribution. In particular, a nonparametric Dirichlet Process Mixture Model (DPMM) with stick-breaking process representation was used. The strength of the DPMM is that it can choose the number of components dynamically as part of the algorithm during parameter estimation. To reduce computational complexity, the modeling process was limited to a maximum of six components. Then, the Markov Chain Monte Carlo (MCMC) sampling technique was employed to estimate the parameters’ posterior distribution. Speed data from nine links of a freeway corridor, aggregated on a 5-minute basis, were used to calculate the corridor travel time. The results demonstrated that this model offers significant flexibility in modeling to account for complex mixture distributions of the travel time without specifying the number of components. The DPMM modeling further revealed that freeway travel time is characterized by multistate or single-state models depending on the inclusion of onset and offset of congestion periods.

A random process called the Dirichlet process whose sample functions are almost surely probability measures has been proposed by Ferguson as an approach to analyzing nonparametric problems from a Bayesian viewpoint. An important result obtained by Ferguson in this approach is that if observations are made on a random variable whose distribution is a random sample function of a Dirichlet process, then the conditional distribution of the random measure can be easily calculated, and is again a Dirichlet process. This paper extends Ferguson's result to cases where the random measure is a mixing distribution for a parameter which determines the distribution from which observations are made. The conditional distribution of the random measure, given the observations, is no longer that of a simple Dirichlet process, but can be described as being a mixture of Dirichlet processes. This paper gives a formal definition for these mixtures and develops several theorems about their properties, the most important of which is a closure property for such mixtures. Formulas for computing the conditional distribution are derived and applications to problems in bio-assay, discrimination, regression, and mixing distributions are given.

We explore spontaneous voltage oscillations in grape must (mustalevria) fermentation systems. This study uses multichannel differential electrode arrays. Seven platinum-iridium (Pt/Ir) electrode pairs tracked bioelectrochemical changes for 200,000 s. They showed complex patterns over time and space. Frequencies varied from 0.00044 to 0.00215 Hz. Power spectral density analysis showed brown noise traits. The spectral slopes ranged from -2.01 to -3.28. This indicates strong temporal integration and memory effects during fermentation. Environmental correlation analysis showed temperature as the primary modulator (r = 0.245-0.558), while humidity exhibited negative correlations (-0.052 to -0.245). Binary state analysis showed that the system uses natural Boolean logic. XOR gates had the highest entropy at 0.93 bits. This suggests that there is significant temporal asynchrony across different spatial areas. Principal component analysis found activation patterns without a single strong mode. It needed 3-4 components to capture 77.6% of the system's variance. The fermentation medium showed uneven metabolic activity across different areas. Also, the electrode locations were statistically independent, with mutual information below 0.206 bits. These findings show that traditional food fermentation systems work like self-organizing bioelectrochemical processors. They can also perform distributed computation through local metabolic interactions. Brown noise scaling and memory effects can impact fermentation monitoring and control. This means short-term measurements may not accurately predict long-term behavior. This work shows that grape must fermentation can be a model system. It helps us study new computational properties in biological electrochemical systems.

We consider the standard Maxwell’s theory in 1+3 dimensions in the presence of a timelike boundary. In this context, we show that (generalized) Ampere-Maxwell’s charge appears as a Noether charge associated with the Maxwell U(1) gauge symmetry which satisfies a spatial conservation equation. Furthermore, we also introduce the notion of spatial memory field and its corresponding memory effect. Finally, similar to the temporal case through the lens of Strominger’s triangle proposal, we show how spatial memory and conservation are related.

A general chemostat microbial cultivation model accounting for the memory effects (in two modifications - μ-type and S-type) is used for biomass and growth rate estimation, based on measurements of the substrate concentration. The influence of the memory effects (history of the process) on the process dynamics is accounted for by employing a zero-order memory functions, characterized by different (in general case) adaptability parameters with respect to the specific growth rate and specific consumption rate. Thus, the memory effects are taken into account in both, growth dynamics and limiting substrate consumption dynamics. Two particular cases, which actually correspond to the most common practical situations on one side, and on the other side - the most suitable model structures of the growth rate from a point of view of control synthesis, are also considered. The first one assumes that the memory effects are taken into account in both, biomass and substrate, equations (specific growth and consumption rates expressions) with equal adaptability parameters, and the second one - in the biomass equation only (specific growth rate expression). The proposed procedure, based on the extended Kalman filtering method, is developed under assumption that the process kinetics models are known except for the adaptability parameter(s). Thus the necessity of performing step-response experiments for the sake of adaptability parameter(s) identification could be successfully avoided. As an example, two estimators, based on different kinetic models of continuous growth of a strain Saccharomyces cerevisiae on a glucose limited medium, are designed. The two models have the same structure and kinetic parameters (identified on the basis of steady-state experiments) and differ the way of memory function incorporation only. The estimation performance is studied under different initial conditions. The effect of the estimation error on the control performance, in case of adaptive control of biomass and substrate concentration, is also investigated. The simulation results are discussed with respect to the applicability of the proposed estimators in the framework of adaptive control systems.

Typically survey data have responses with gaps, outliers and ties, and the distributions of the responses might be skewed. Usually, in small area estimation, predictive inference is done using a two-stage Bayesian model with normality at both levels (responses and area means). This is the Scott-Smith (S-S) model and it may not be robust against these features. Another model that can be used to provide a more robust structure is the two-stage Dirichlet process mixture (DPM) model, which has independent normal distributions on the responses and a single Dirichlet process on the area means. However, this model does not accommodate gaps, outliers and ties in the survey data directly. Because this DPM model has a normal distribution on the responses, it is unlikely to be realized in practice, and this is the problem we tackle in this paper. Therefore, we propose a two-stage non-parametric Bayesian model with several independent Dirichlet processes at the first stage that represents the data, thereby accommodating some of the difficulties with survey data and permitting a more robust predictive inference. This model has a Gaussian (normal) distribution on the area means, and so we call it the DPG model. Therefore, the DPM model and the DPG model are essentially the opposite of each other and they are both different from the S-S model. Among the three models, the DPG model gives us the best head-start to accommodate the features of the survey data. For Bayesian predictive inference, we need to integrate two data sets, one with the responses and other with area sizes. An application on body mass index, which is integrated with census data, and a simulation study are used to compare the three models (S-S, DPM, DPG); we show that the DPG model might be preferred.