Chinese Restaurant Process Research Articles

Branch merging on continuum trees with applications to regenerative tree growth

We introduce a family of branch merging operations on continuum trees and show that Ford CRTs are distributionally invariant. This operation is new even in the special case of the Brownian CRT, which we explore in more detail. The operations are based on spinal decompositions and a regenerativity preserving merging procedure of $(\alpha, \theta)$-strings of beads, that is, random intervals $[0, L_{\alpha, \theta}]$ equipped with a random discrete measure $dL^{-1}$ arising in the limit of ordered $(\alpha, \theta)$-Chinese restaurant processes as introduced recently by Pitman and Winkel. Indeed, we iterate the branch merging operation recursively and give an alternative approach to the leaf embedding problem on Ford CRTs related to $(\alpha, 2-\alpha)$-regenerative tree growth processes.

On moment sequences and mixed Poisson distributions

In this article we survey properties of mixed Poisson distributions and probabilistic aspects of the Stirling transform: given a non-negative random variable (X) with moment sequence ((mu_s)_{sinmathbb{N}}) we determine a discrete random variable (Y), whose moment sequence is given by the Stirling transform of the sequence ((mu_s)_{sinmathbb{N}}), and identify the distribution as a mixed Poisson distribution. We discuss properties of this family of distributions and present a new simple limit theorem based on expansions of factorial moments instead of power moments. Moreover, we present several examples of mixed Poisson distributions in the analysis of random discrete structures, unifying and extending earlier results. We also add several entirely new results: we analyse triangular urn models, where the initial configuration or the dimension of the urn is not fixed, but may depend on the discrete time (n). We discuss the branching structure of plane recursive trees and its relation to table sizes in the Chinese restaurant process. Furthermore, we discuss root isolation procedures in Cayley trees, a parameter in parking functions, zero contacts in lattice paths consisting of bridges, and a parameter related to cyclic points and trees in graphs of random mappings, all leading to mixed Poisson-Rayleigh distributions. Finally, we indicate how mixed Poisson distributions naturally arise in the critical composition scheme of Analytic Combinatorics. <script type=text/javascript src=//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML>

Tracklet clustering for robust multiple object tracking using distance dependent Chinese restaurant processes

To contrive an accurate and efficient strategy for object detection–object track assignment problem, we present a tracklet clustering approach using distance dependent Chinese restaurant processes (ddCRPs), which employ a two-level robust object tracker. The first level is an ordinary tracklet generator that obtains short yet reliable tracklets. In the second level, we cluster the tracklets over time based on color, spatial and temporal attributes, where the nonparametric process of clustering with ddCRPs allows us to maintain an unknown number of objects. Unlike the previously proposed Chinese restaurant processes and Dirichlet process mixture models, our ddCRPs method does not require prescribed complex cluster models to be initialized and updated, and thus, we can cluster complex tracklets by only computing similarities between them. Our comparative evaluations on tracking different object types demonstrate the generality of our approach.

Device-to-Device Users Clustering Based on Physical and Social Characteristics

This paper proposes a novel method for device-to-device (D2D) user clustering that allows wireless users in proximity to share common resources to save both system bandwidth and energy resources. The idea at the basis of our proposed cluster formation is to incorporate both social interactions and physical relationships among D2D terminals. Towards this goal, we propose two clustering approaches. The first one is based on the Chinese restaurant process (CRP), whereas the second one enhances the traditional CRP by defining a “distance-dependent Chinese restaurant process” (namely, DCRP). Numerical simulation results demonstrate superiority of our proposed clustering schemes in terms of system throughput, energy consumption, and energy efficiency over the existing schemes that rely only on physical distance.

Hierarchical Pitman–Yor–Dirichlet Language Model

Probabilistic models are often viewed as insufficiently expressive because of strong limitation and assumption on the probabilistic distribution and the fixed model complexity. Bayesian nonparametric learning pursues an expressive probabilistic representation based on the nonparametric prior and posterior distributions with less assumption-laden approach to inference. This paper presents a hierarchical Pitman-Yor-Dirichlet (HPYD) process as the nonparametric priors to infer the predictive probabilities of the smoothed n-grams with the integrated topic information. A metaphor of hierarchical Chinese restaurant process is proposed to infer the HPYD language model (HPYD-LM) via Gibbs sampling. This process is equivalent to implement the hierarchical Dirichlet process-latent Dirichlet allocation (HDP-LDA) with the twisted hierarchical Pitman-Yor LM (HPY-LM) as base measures. Accordingly, we produce the power-law distributions and extract the semantic topics to reflect the properties of natural language in the estimated HPYD-LM. The superiority of HPYD-LM to HPY-LM and other language models is demonstrated by the experiments on model perplexity and speech recognition.

Discovering hierarchical topic evolution in time‐stamped documents

The objective of this paper is to propose a hierarchical topic evolution model (HTEM) that can organize time‐varying topics in a hierarchy and discover their evolutions with multiple timescales. In the proposed HTEM, topics near the root of the hierarchy are more abstract and also evolve in the longer timescales than those near the leaves. To achieve this goal, the distance‐dependent Chinese restaurant process (ddCRP) is extended to a new nested process that is able to simultaneously model the dependencies among data and the relationship between clusters. The HTEM is proposed based on the new process for time‐stamped documents, in which the timestamp is utilized to measure the dependencies among documents. Moreover, an efficient Gibbs sampler is developed for the proposed HTEM. Our experimental results on two popular real‐world data sets verify that the proposed HTEM can capture coherent topics and discover their hierarchical evolutions. It also outperforms the baseline model in terms of likelihood on held‐out data.

Infinite Sparse Block Model with Text Using 2DCRP

We present a new approach to estimating the interdependence of industries in an economy by applying data science solutions. By exploiting interfirm buyer--seller network data, we show that the problem of estimating the interdependence of industries is similar to the problem of uncovering the latent block structure in network science literature. To estimate the underlying structure with greater accuracy, we propose an extension of the sparse block model that incorporates node textual information and an unbounded number of industries and interactions among them. The latter task is accomplished by extending the well-known Chinese restaurant process to two dimensions. Inference is based on collapsed Gibbs sampling, and the model is evaluated on both synthetic and real-world datasets. We show that the proposed model improves in predictive accuracy and successfully provides a satisfactory solution to the motivated problem. We also discuss issues that affect the future performance of this approach.

Bayesian clustering of shapes of curves

Unsupervised clustering of curves according to their shapes is an important problem with broad scientific applications. The existing model-based clustering techniques either rely on simple probability models (e.g., Gaussian) that are not generally valid for shape analysis or assume the number of clusters. We develop an efficient Bayesian method to cluster curve data using an elastic shape metric that is based on joint registration and comparison of shapes of curves. The elastic-inner product matrix obtained from the data is modeled using a Wishart distribution whose parameters are assigned carefully chosen prior distributions to allow for automatic inference on the number of clusters. Posterior is sampled through an efficient Markov chain Monte Carlo procedure based on the Chinese restaurant process to infer (1) the posterior distribution on the number of clusters, and (2) clustering configuration of shapes. This method is demonstrated on a variety of synthetic data and real data examples on protein structure analysis, cell shape analysis in microscopy images, and clustering of shapes from MPEG7 database.

Data clustering using side information dependent Chinese restaurant processes

Side information, or auxiliary information associated with documents or image content, provides hints for clustering. We propose a new model, side information dependent Chinese restaurant process, which exploits side information in a Bayesian nonparametric model to improve data clustering. We introduce side information into the framework of distance dependent Chinese restaurant process using a robust decay function to handle noisy side information. The threshold parameter of the decay function is updated automatically in the Gibbs sampling process. A fast inference algorithm is proposed. We evaluate our approach on four datasets: Cora, 20 Newsgroups, NUS-WIDE and one medical dataset. Types of side information explored in this paper include citations, authors, tags, keywords and auxiliary clinical information. The comparison with the state-of-the-art approaches based on standard performance measures (NMI, F1) clearly shows the superiority of our approach.

The Dynamic Chinese Restaurant Process via Birth and Death Processes

We develop the Dynamic Chinese Restaurant Process (DCRP) which incorporates time-evolutionary feature in dependent Dirichlet Process mixture models. This model can capture the dynamic change of mixture components, allowing clusters to emerge, vanish and vary over time. All these macroscopic changes are controlled by tracing the birth and death of every single element. We investigate the properties of dependent Dirichlet Process mixture model based on DCRP and develop corresponding Gibbs Sampler for posterior inference. We also conduct simulation and empirical studies to compare this model with traditional CRP and related models. The results show that this model can provide better results for sequential data, especially for data with heterogeneous lifetime distribution.

Pitman Yor Diffusion Trees for Bayesian Hierarchical Clustering.

In this paper we introduce the Pitman Yor Diffusion Tree (PYDT), a Bayesian non-parametric prior over tree structures which generalises the Dirichlet Diffusion Tree [30] and removes the restriction to binary branching structure. The generative process is described and shown to result in an exchangeable distribution over data points. We prove some theoretical properties of the model including showing its construction as the continuum limit of a nested Chinese restaurant process model. We then present two alternative MCMC samplers which allow us to model uncertainty over tree structures, and a computationally efficient greedy Bayesian EM search algorithm. Both algorithms use message passing on the tree structure. The utility of the model and algorithms is demonstrated on synthetic and real world data, both continuous and binary.

Clustering disaggregated load profiles using a Dirichlet process mixture model

The increasing availability of substantial quantities of power-use data in both the residential and commercial sectors raises the possibility of mining the data to the advantage of both consumers and network operations. We present a Bayesian non-parametric model to cluster load profiles from households and business premises. Evaluators show that our model performs as well as other popular clustering methods, but unlike most other methods it does not require the number of clusters to be predetermined by the user. We used the so-called ‘Chinese restaurant process’ method to solve the model, making use of the Dirichlet-multinomial distribution. The number of clusters grew logarithmically with the quantity of data, making the technique suitable for scaling to large data sets. We were able to show that the model could distinguish features such as the nationality, household size, and type of dwelling between the cluster memberships.

Bayesian nonparametric crowdsourcing

Crowdsourcing has been proven to be an effective and efficient tool to annotate large data-sets. User annotations are often noisy, so methods to combine the annotations to produce reliable estimates of the ground truth are necessary. We claim that considering the existence of clusters of users in this combination step can improve the performance. This is especially important in early stages of crowdsourcing implementations, where the number of annotations is low. At this stage there is not enough information to accurately estimate the bias introduced by each annotator separately, so we have to resort to models that consider the statistical links among them. In addition, finding these clusters is interesting in itself as knowing the behavior of the pool of annotators allows implementing efficient active learning strategies. Based on this, we propose in this paper two new fully unsupervised models based on a Chinese restaurant process (CRP) prior and a hierarchical structure that allows inferring these groups jointly with the ground truth and the properties of the users. Efficient inference algorithms based on Gibbs sampling with auxiliary variables are proposed. Finally, we perform experiments, both on synthetic and real databases, to show the advantages of our models over state-of-the-art algorithms.

Non-parametric topic model for discovering geographical topic variations

This paper presents a non-parametric topic model that captures not only the latent topics in text collections, but also how the topics change over space. Unlike other recent work that relies on either Gaussian assumptions or discretization of locations, here topics are associated with a distance dependent Chinese Restaurant Process (ddCRP), and for each document, the observed words are influenced by the document’s GPS-tag. Our model allows both unbound number and flexible distribution of the geographical variations of the topics’ content. We develop a Gibbs sampler for the proposal, and compare it with existing models on a real data set basis.

Redundancy of Exchangeable Estimators

Exchangeable random partition processes are the basis for Bayesian approaches to statistical inference in large alphabet settings. On the other hand, the notion of the pattern of a sequence provides an information-theoretic framework for data compression in large alphabet scenarios. Because data compression and parameter estimation are intimately related, we study the redundancy of Bayes estimators coming from Poisson–Dirichlet priors (or “Chinese restaurant processes”) and the Pitman–Yor prior. This provides an understanding of these estimators in the setting of unknown discrete alphabets from the perspective of universal compression. In particular, we identify relations between alphabet sizes and sample sizes where the redundancy is small, thereby characterizing useful regimes for these estimators.

Multilevel Chinese takeaway process and label-based processes for rule induction in the context of automated sports video annotation.

We propose four variants of a novel hierarchical hidden Markov models strategy for rule induction in the context of automated sports video annotation including a multilevel Chinese takeaway process (MLCTP) based on the Chinese restaurant process and a novel Cartesian product label-based hierarchical bottom-up clustering (CLHBC) method that employs prior information contained within label structures. Our results show significant improvement by comparison against the flat Markov model: optimal performance is obtained using a hybrid method, which combines the MLCTP generated hierarchical topological structures with CLHBC generated event labels. We also show that the methods proposed are generalizable to other rule-based environments including human driving behavior and human actions.

Bayesian Multiscale Smoothing of Gaussian Noised Images

We propose a multiscale model for Gaussian noised images under a Bayesian framework for both 2-dimensional (2D) and 3-dimensional (3D) images. We use a Chinese restaurant process prior to randomly generate ties among intensity values at neighboring pixels in the image. The resulting Bayesian estimator enjoys some desirable asymptotic properties for identifying precise structures in the image. The proposed Bayesian denoising procedure is completely data-driven. A conditional conjugacy property allows analytical computation of the posterior distribution without involving Markov chain Monte Carlo (MCMC) methods, making the method computationally efficient. Simulations on Shepp-Logan phantom and Lena test images confirm that our smoothing method is comparable with the best available methods for light noise and outperforms them for heavier noise both visually and numerically. The proposed method is further extended for 3D images. A simulation study shows that the proposed method is numerically better than most existing denoising approaches for 3D images. A 3D Shepp-Logan phantom image is used to demonstrate the visual and numerical performance of the proposed method, along with the computational time. MATLAB toolboxes are made available online (both 2D and 3D) to implement the proposed method and reproduce the numerical results.

Minimal Clade Size in the Bolthausen-Sznitman Coalescent

In this article we show the asymptotics of distribution and moments of the sizeXnof the minimal clade of a randomly chosen individual in a Bolthausen-Sznitmann-coalescent forn→ ∞. The Bolthausen-Sznitmann-coalescent is a Markov process taking states in the set of partitions of {1, …,n}, where 1, …,nare referred to as individuals. The minimal clade of an individual is the equivalence class the individual is in at the time of the first coalescence event this individual participates in. We also provide exact formulae for the distribution ofXn. The main tool used is the connection of the Bolthausen-Sznitmann-coalescent with random recursive trees introduced by Goldschmidt and Martin (2005). With it, we show thatXn- 1 is distributed as the size of a uniformly chosen table in a standard Chinese restaurant process withn- 1 customers.

Minimal Clade Size in the Bolthausen-Sznitman Coalescent

In this article we show the asymptotics of distribution and moments of the sizeXnof the minimal clade of a randomly chosen individual in a Bolthausen-Sznitmann-coalescent forn→ ∞. The Bolthausen-Sznitmann-coalescent is a Markov process taking states in the set of partitions of {1, …,n}, where 1, …,nare referred to as individuals. The minimal clade of an individual is the equivalence class the individual is in at the time of the first coalescence event this individual participates in. We also provide exact formulae for the distribution ofXn. The main tool used is the connection of the Bolthausen-Sznitmann-coalescent with random recursive trees introduced by Goldschmidt and Martin (2005). With it, we show thatXn- 1 is distributed as the size of a uniformly chosen table in a standard Chinese restaurant process withn- 1 customers.

Cycles and eigenvalues of sequentially growing random regular graphs

Consider the sum of $d$ many i.i.d. random permutation matrices on $n$ labels along with their transposes. The resulting matrix is the adjacency matrix of a random regular (multi)-graph of degree $2d$ on $n$ vertices. It is known that the distribution of smooth linear eigenvalue statistics of this matrix is given asymptotically by sums of Poisson random variables. This is in contrast with Gaussian fluctuation of similar quantities in the case of Wigner matrices. It is also known that for Wigner matrices the joint fluctuation of linear eigenvalue statistics across minors of growing sizes can be expressed in terms of the Gaussian Free Field (GFF). In this article, we explore joint asymptotic (in $n$) fluctuation for a coupling of all random regular graphs of various degrees obtained by growing each component permutation according to the Chinese Restaurant Process. Our primary result is that the corresponding eigenvalue statistics can be expressed in terms of a family of independent Yule processes with immigration. These processes track the evolution of short cycles in the graph. If we now take $d$ to infinity, certain GFF-like properties emerge.