Stick-breaking Representation Research Articles

Nonparametric Bayesian optimal designs for Unit Exponential regression model with respect to prior processes(with the truncated normal as the base measure)

Nonlinear regression models are extensively applied across various scientific disciplines. It is vital to accurately fit the optimal nonlinear model while considering the biases of the Bayesian optimal design. We present a Bayesian optimal design by utilising the Dirichlet process as a prior. The Dirichlet process serves as a fundamental tool in the exploration of Nonparametric Bayesian inference, offering multiple representations that are well-suited for application. This research paper introduces a novel one-parameter model, referred to as the ’Unit-Exponential distribution’, specifically designed for the unit interval. Additionally, we employ a stick-breaking representation to approximate the D-optimality criterion considering the Dirichlet process as a functional tool. Through this approach, we aim to identify a Nonparametric Bayesian optimal design.

Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis.

The paper discusses shrinkage priors which impose increasing shrinkage in a sequence of parameters. We review the cumulative shrinkage process (CUSP) prior of Legramanti et al. (Legramanti et al. 2020 Biometrika 107, 745-752. (doi:10.1093/biomet/asaa008)), which is a spike-and-slab shrinkage prior where the spike probability is stochastically increasing and constructed from the stick-breaking representation of a Dirichlet process prior. As a first contribution, this CUSP prior is extended by involving arbitrary stick-breaking representations arising from beta distributions. As a second contribution, we prove that exchangeable spike-and-slab priors, which are popular and widely used in sparse Bayesian factor analysis, can be represented as a finite generalized CUSP prior, which is easily obtained from the decreasing order statistics of the slab probabilities. Hence, exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index in the loading matrix increases, without imposing explicit order constraints on the slab probabilities. An application to sparse Bayesian factor analysis illustrates the usefulness of the findings of this paper. A new exchangeable spike-and-slab shrinkage prior based on the triple gamma prior of Cadonna et al. (Cadonna et al. 2020 Econometrics 8, 20. (doi:10.3390/econometrics8020020)) is introduced and shown to be helpful for estimating the unknown number of factors in a simulation study. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

Asymptotic shape of the concave majorant of a Lévy process

We establish distributional limit theorems for the shape statistics of a concave majorant (i.e. the fluctuations of its length, its supremum, the time it is attained and its value at $T$) of any Levy process on $[0,T]$ as $T\to\infty$. The scale of the fluctuations of the length and other statistics, as well as their asymptotic dependence, vary significantly with the tail behaviour of the Levy measure. The key tool in the proofs is the recent representation of the concave majorant for all Levy processes using a stick-breaking representation.

Association between air pollution and COVID-19 disease severity via Bayesian multinomial logistic regression with partially missing outcomes.

Recent ecological analyses suggest air pollution exposure may increase susceptibility to and severity of coronavirus disease 2019 (COVID‐19). Individual‐level studies are needed to clarify the relationship between air pollution exposure and COVID‐19 outcomes. We conduct an individual‐level analysis of long‐term exposure to air pollution and weather on peak COVID‐19 severity. We develop a Bayesian multinomial logistic regression model with a multiple imputation approach to impute partially missing health outcomes. Our approach is based on the stick‐breaking representation of the multinomial distribution, which offers computational advantages, but presents challenges in interpreting regression coefficients. We propose a novel inferential approach to address these challenges. In a simulation study, we demonstrate our method's ability to impute missing outcome data and improve estimation of regression coefficients compared to a complete case analysis. In our analysis of 55,273 COVID‐19 cases in Denver, Colorado, increased annual exposure to fine particulate matter in the year prior to the pandemic was associated with increased risk of severe COVID‐19 outcomes. We also found COVID‐19 disease severity to be associated with interactions between exposures. Our individual‐level analysis fills a gap in the literature and helps to elucidate the association between long‐term exposure to air pollution and COVID‐19 outcomes.

Bayesian multivariate quantile regression using Dependent Dirichlet Process prior

In this article, we consider a non-parametric Bayesian approach to multivariate quantile regression. The proposed approach involves modeling of related conditional distributions of a response vector given the covariates using a Dependent Dirichlet Process (DDP) prior. The DDP is used to introduce dependence across covariates. The flexible covariate-dependent mixture of multivariate Gaussian kernels gives rise to an induced posterior for the desired multivariate quantile. For posterior computations, we use a truncated stick-breaking representation of the DDP, and use a block Gibbs sampler for estimating the model parameters. We illustrate our method with simulation studies, and a data containing blood pressures of 40 women. Finally, we provide a theoretical justification for the proposed method through posterior consistency and support properties of the prior.

Stick-Breaking Dependent Beta Processes with Variational Inference

The beta processes (BP) is a powerful nonparametric tool in feature learning, which is often used as the prior of Bernoulli process for choosing features from a feature dictionary. However, it still shows a limitation in processing some real-world data, as the stood and BP is independent of data. In practice, the probabilities of selecting features in the latent space for different observed data are different, and they are usually dependent on some information from data, such as the location or time information. For example, data with closer distances usually have similar features. This kind of information (location or time) often called covariates, which are ignored in most BP-related literature. To account this problem, we propose a variational inference based dependent beta processes (VDBP), in which the dependent beta process is constructed using the stick-breaking representation and the dependency on the covariates is captured by a Gaussian process prior. An elegant representation of variational inference for with VDBP prior is obtained, which offers the efficient training method for the models using VDBP as priors. Through instantiating a Bayesian factor analysis model with VDBP, we verify the effectiveness of the proposed VDBP in image denoising and image inpainting tasks.

Nonparametric Bayesian Modeling and Estimation of Spatial Correlation Functions for Global Data

We provide a nonparametric spectral approach to the modeling of correlation functions on spheres. The sequence of Schoenberg coefficients and their associated covariance functions are treated as random rather than assuming a parametric form. We propose a stick-breaking representation for the spectrum, and show that such a choice spans the support of the class of geodesically isotropic covariance functions under uniform convergence. Further, we examine the first order properties of such representation, from which geometric properties can be inferred, in terms of Hölder continuity, of the associated Gaussian random field. The properties of the posterior, in terms of existence, uniqueness, and Lipschitz continuity, are then inspected. Our findings are validated with MCMC simulations and illustrated using a global data set on surface temperatures.

Bayesian nonparametric priors for hidden Markov random fields

One of the central issues in statistics and machine learning is how to select an adequate model that can automatically adapt its complexity to the observed data. In the present paper, we focus on the issue of determining the structure of clustered data, both in terms of finding the appropriate number of clusters and of modelling the right dependence structure between the observations. Bayesian nonparametric (BNP) models, which do not impose an upper limit on the number of clusters, are appropriate to avoid the required guess on the number of clusters but have been mainly developed for independent data. In contrast, Markov random fields (MRF) have been extensively used to model dependencies in a tractable manner but usually reduce to finite cluster numbers when clustering tasks are addressed. Our main contribution is to propose a general scheme to design tractable BNP-MRF priors that combine both features: no commitment to an arbitrary number of clusters and a dependence modelling. A key ingredient in this construction is the availability of a stick-breaking representation which has the threefold advantage to allowing us to extend standard discrete MRFs to infinite state space, to design a tractable estimation algorithm using variational approximation and to derive theoretical properties on the predictive distribution and the number of clusters of the proposed model. This approach is illustrated on a challenging natural image segmentation task for which it shows good performance with respect to the literature.

A new proof of the stick-breaking representation of Dirichlet processes

The stick-breaking representation is one of the fundamental properties of the Dirichlet process. It represents the random probability measure as a discrete random sum whose weights and atoms are formed by independent and identically distributed sequences of beta variates and draws from the normalized base measure of the Dirichlet process parameter. It is used extensively in posterior simulation for statistical models with Dirichlet processes. The original proof of Sethuraman (Stat Sin 4:639–650, 1994) relies on an indirect distributional equation and does not encourage an intuitive understanding of the property. In this paper, we give a new proof of the stick-breaking representation of the Dirichlet process that provides an intuitive understanding of the theorem. The proof is based on the posterior distribution and self-similarity properties of the Dirichlet process.

Nonparametric prior elicitation for a binomial proportion

This paper proposes a nonparametric Bayesian approach based on a density estimation with an open unit interval (0,1) using binomial data. We propose a very efficient nonparametric Bayesian approach method to infer smooth density defined on (0,1) through the transformation of a random variable. For practical implementation, we provide the corresponding blocked Gibbs sampling procedure based on the stick-breaking representation. The greatest advantage of this method is that it does not require us to draw from the complete conditional posterior distribution using a Metropolis-Hastings transition probability because the proposed transformation leads to a pair of conjugate priors and likelihoods. The validity of the proposed method is assessed through simulated and real data analysis.

A simple proof of Pitman–Yor’s Chinese restaurant process from its stick-breaking representation

Abstract For a long time, the Dirichlet process has been the gold standard discrete random measure in Bayesian nonparametrics. The Pitman-Yor process provides a simple and mathematically tractable generalization, allowing for a very flexible control of the clustering behaviour. Two commonly used representations of the Pitman-Yor process are the stick-breaking process and the Chinese restaurant process. The former is a constructive representation of the process which turns out very handy for practical implementation, while the latter describes the partition distribution induced. Obtaining one from the other is usually done indirectly with use of measure theory. In contrast, we propose here an elementary proof of Pitman-Yor’s Chinese Restaurant process from its stick-breaking representation.

Stochastic Approximations to the Pitman–Yor Process

In this paper we consider approximations to the popular Pitman–Yor process obtained by truncating the stick-breaking representation. The truncation is determined by a random stopping rule that achieves an almost sure control on the approximation error in total variation distance. We derive the asymptotic distribution of the random truncation point as the approximation error ϵ goes to zero in terms of a polynomially tilted positive stable random variable. The practical usefulness and effectiveness of this theoretical result is demonstrated by devising a sampling algorithm to approximate functionals of the ϵ-version of the Pitman–Yor process.

Axially Symmetric Data Clustering Through Dirichlet Process Mixture Models of Watson Distributions.

This paper proposes a Bayesian nonparametric framework for clustering axially symmetric data. Our approach is based on a Dirichlet processes mixture model with Watson distributions, which can also be considered as the infinite Watson mixture model. In this paper, first, we extend the finite Watson mixture model into its infinite counterpart based on the framework of truncated Dirichlet process mixture model with a stick-breaking representation. Second, we propose a coordinate ascent mean-field variational inference algorithm that can effectively learn the parameters of our model with closed-form solutions; Third, to cope with a massive data set, we develop a stochastic variational inference algorithm to learn the proposed model through the method of stochastic gradient ascent; Finally, the proposed nonparametric Bayesian model is evaluated through simulated axially symmetric data sets and a real-world application, namely, gene expression data clustering.

Mixture Models With a Prior on the Number of Components

ABSTRACTA natural Bayesian approach for mixture models with an unknown number of components is to take the usual finite mixture model with symmetric Dirichlet weights, and put a prior on the number of components—that is, to use a mixture of finite mixtures (MFM). The most commonly used method of inference for MFMs is reversible jump Markov chain Monte Carlo, but it can be nontrivial to design good reversible jump moves, especially in high-dimensional spaces. Meanwhile, there are samplers for Dirichlet process mixture (DPM) models that are relatively simple and are easily adapted to new applications. It turns out that, in fact, many of the essential properties of DPMs are also exhibited by MFMs—an exchangeable partition distribution, restaurant process, random measure representation, and stick-breaking representation—and crucially, the MFM analogues are simple enough that they can be used much like the corresponding DPM properties. Consequently, many of the powerful methods developed for inference in DPMs can be directly applied to MFMs as well; this simplifies the implementation of MFMs and can substantially improve mixing. We illustrate with real and simulated data, including high-dimensional gene expression data used to discriminate cancer subtypes. Supplementary materials for this article are available online.

Variational Bayesian inference for a Dirichlet process mixture of beta distributions and application

Abstract Finite beta mixture model (BMM) has been shown to be very flexible and powerful for bounded support data modeling. However, BMM cannot automatically select the proper number of the mixture components based on the observed data, which is important and has a deterministic effect on the modeling accuracy. In this paper, we aim at tackling this problem by infinite Beta mixture model (InBMM). It is based on the Dirichlet process (DP) mixture with the assumption that the number of the mixture components is infinite in advance and can be automatically determined according to the observed data. Further, a variational InBMM using single lower-bound approximation (VBInBMM) is proposed which applies the stick-breaking representation of the DP and is learned by an extended variational inference framework. Numerical experiments on both synthetic and real data, generated from two challenging application namely image categorization and object detection, demonstrate good performance obtained by the proposed method.

Bayesian Nonparametric Model for Estimating Multistate Travel Time Distribution

Multistate models, that is, models with more than two distributions, are preferred over single-state probability models in modeling the distribution of travel time. Literature review indicated that the finite multistate modeling of travel time using lognormal distribution is superior to other probability functions. In this study, we extend the finite multistate lognormal model of estimating the travel time distribution to unbounded lognormal distribution. In particular, a nonparametric Dirichlet Process Mixture Model (DPMM) with stick-breaking process representation was used. The strength of the DPMM is that it can choose the number of components dynamically as part of the algorithm during parameter estimation. To reduce computational complexity, the modeling process was limited to a maximum of six components. Then, the Markov Chain Monte Carlo (MCMC) sampling technique was employed to estimate the parameters’ posterior distribution. Speed data from nine links of a freeway corridor, aggregated on a 5-minute basis, were used to calculate the corridor travel time. The results demonstrated that this model offers significant flexibility in modeling to account for complex mixture distributions of the travel time without specifying the number of components. The DPMM modeling further revealed that freeway travel time is characterized by multistate or single-state models depending on the inclusion of onset and offset of congestion periods.

On the Stick-Breaking Representation for Homogeneous NRMIs

In this paper, we consider homogeneous normalized random measures with independent increments (hNRMI), a class of nonparametric priors recently introduced in the literature. Many of their distributional properties are known by now but their stick-breaking representation is missing. Here we display such a representation, which will feature dependent stick-breaking weights, and then derive explicit versions for noteworthy special cases of hNRMI. Posterior charac- terizations are also discussed. Finally, we devise an algorithm for slice sampling mixture models based on hNRMIs, which relies on the representation we have obtained, and implement it to analyze real data.

Dynamic density estimation with diffusive Dirichlet mixtures

We introduce a new class of nonparametric prior distributions on the space of continuously varying densities, induced by Dirichlet process mixtures which diffuse in time. These select time-indexed random functions without jumps, whose sections are continuous or discrete distributions depending on the choice of kernel. The construction exploits the widely used stick-breaking representation of the Dirichlet process and induces the time dependence by replacing the stick-breaking components with one-dimensional Wright-Fisher diffusions. These features combine appealing properties of the model, inherited from the Wright-Fisher diffusions and the Dirichlet mixture structure, with great flexibility and tractability for posterior computation. The construction can be easily extended to multi-parameter GEM marginal states, which include, for example, the Pitman--Yor process. A full inferential strategy is detailed and illustrated on simulated and real data.

Stick-Breaking Representation and Computation for Normalized Generalized Gamma Processes

Using fractions of gamma and exponentially titled stable random variables this article develops a stick-breaking representation of a truncated normalized generalized gamma process. Sampling from the posterior of this process requires sampling from gamma titled stable random variables and we develop an algorithm to do so that is readily implemented in the open source software R. A Blocked Gibbs sampling algorithm for a Bayesian kernel mixture model is then described and we compare the performance of our algorithm with an algorithm recently proposed in the literature.

Adaptive Bayesian Density Estimation in Lp-metrics with Pitman-Yor or Normalized Inverse-Gaussian Process Kernel Mixtures

We consider Bayesian nonparametric density estimation using a Pitman-Yor or a normalized inverse-Gaussian process convolution kernel mixture as the prior distribution for a density. The procedure is studied from a frequentist perspective. Using the stick-breaking representation of the Pitman-Yor process and the finite-dimensional distributions of the normalized inverse-Gaussian process, we prove that, when the data are independent replicates from a density with analytic or Sobolev smoothness, the posterior distribution concentrates on shrinking Lp-norm balls around the sampling density at a minimax-optimal rate, up to a logarithmic factor. The resulting hierarchical Bayesian procedure, with a fixed prior, is adaptive to the unknown smoothness of the sampling density.