Dirichlet and Related Processes

Abstract

The Dirichlet process plays a dominant role as a prior in Bayesian nonparametrics leading to the development of a wide variety of inferential procedures. In this chapter we give a comprehensive account of the Dirichlet process and its immediate variants—the Dirichlet invariant and mixture of Dirichlet processes. Starting with its formal definition and alternative representations which include the seminal Sethuraman representation, we proceed to discuss in depth many of its properties including the most important one, the conjugacy property, which makes posterior computations easy by simply updating its parameter. Its marginal distribution is characterized as the generalized Polya urn scheme, also known as the Chinese restaurant process, and is shown to provide a tool to sample the Dirichlet process easily. We identify a major limitation that with probability one, it selects a random discrete distribution. However, it is shown in the next chapter that the discreteness proves to be a useful feature in formulating a variety of Dirichlet mixture models. Next, we present the Dirichlet invariant and symmetrized Dirichlet prior processes which are found to be appropriate when the unknown distribution function is known a priori to be is structured such as being invariant under a finite group of transformations or being symmetric.To address the inadequacy of the Dirichlet process in dealing with certain types of data such as bioassay or right censored data, a mixture of Dirichlet processes is developed by treating the parameter of the Dirichlet process as random having a certain parametric distribution. This is described next along with its various properties. The mixtures of Dirichlet processes are shown to be extremely useful in developing various hierarchical and mixture models, some of which are presented in this chapter and others in the next chapter. A major hurdle in implementation of these models is that the posterior distributions needed to proceed with the Bayesian analysis are often found to be intractable making it necessary to generate them via simulation. Consequently, various computational procedures based on Monte Carlo Markov chain developed in the literature using Gibbs sampler, blocked Gibbs samples, slice and retrospective sampling are described here sparingly and relevant steps of algorithms proposed by the respective authors are included here for ease of understanding. Finally, other extensions of the Dirichlet process such as multinomial Dirichlet process, multivariate and generalized Dirichlet processes are briefly outlined.