Priors Based on Levy Processes

Abstract

The independent increments process, also known as (positive) Levy process, with its associated Levy measure has emerged as an important vehicle in the development of a group of processes such as neutral to the right, gamma, extended gamma, beta, and beta-Stacy processes that are discussed cohesively in this chapter. After a brief review of the Levy process and its generalization the completely random measure and of the Poisson Process which is used in its construction, we present a detail treatment of the neutral to the right process defined on the real line since all other processes discussed here can be classified as of the neutral to the right type. It is shown to have the desirable conjugacy property which facilitates the Bayesian analysis of statistical data even in the case of right censored data which the Dirichlet process fails to do. We devote a sizable effort to the derivation of the posterior distribution under this prior and simulation procedures necessary thereof, since it carries over to other neutral to the right type processes as well. Thereafter we introduce a simple case of Levy process, namely the gamma process, in which the independent increments are assumed to be distributed as gamma distribution, followed by its convolution with a known function, the extended gamma process, to serve as a prior on the space of hazard functions.We discuss next the beta process, which was developed to place a prior on the space of cumulative hazard functions defined on the real line, and also indirectly on the space of distribution functions, along with its important properties. The Levy measure of the beta process when viewed on an arbitrary space is shown to be useful in generating a prior process known as the Indian buffet process, which serves as a prior on the space of sparse binary matrices encountered in featural modeling. Further generalizations of the beta process, such as the stable and kernel beta processes, are also considered in the same context. A special case of the Levy process—the log-beta process, in which the increments are assumed to have a beta-Stacy distribution, is used in developing the beta-Stacy process presented next, and noted that the latter not only generalizes the Dirichlet process but is also conjugate with respect to data which may include right censored observations as well. Finally, we present the derivation of three special processes, namely the Chinese restaurant, the Indian buffet, and infinite gamma-Poisson processes, which have emerged to be very useful in areas outside the statistical field, such as machine learning, information retrieval, and featural modeling. We also discuss in the chapter various properties, special characteristics, essential features and limitations, as well as challenges in applications of the above mentioned processes.