Yor Process Research Articles

A nonparametric Bayesian learning model using accelerated variational inference and feature selection

Developing effective machine learning methods for multimedia data modeling continues to challenge computer vision scientists. The capability of providing effective learning models can have significant impact on various applications. In this work, we propose a nonparametric Bayesian approach to address simultaneously two fundamental problems, namely clustering and feature selection. The approach is based on infinite generalized Dirichlet (GD) mixture models constructed through the framework of Dirichlet process and learned using an accelerated variational algorithm that we have developed. Furthermore, we extend the proposed approach using another nonparametric Bayesian prior, namely Pitman---Yor process, to construct the infinite generalized Dirichlet mixture model. Our experiments, which were conducted through synthetic data sets, the clustering analysis of real-world data sets and a challenging application, namely automatic human action recognition, indicate that the proposed framework provides good modeling and generalization capabilities.

Simultaneous clustering and feature selection via nonparametric Pitman–Yor process mixture models

Mixture models constitute one of the most important machine learning approaches. Indeed, they can be considered as the workhorse of generative machine learning. The majority of existing works consider mixtures of Gaussians. Unlike these works, this paper concentrates on nonparametric Bayesian models with Dirichlet-based mixtures. In particular, we consider the case when a Pitman–Yor process prior is adopted. Two central problems when considering such mixtures can be regarded as selecting ‘meaningful’ (or relevant) features and estimating the model’s parameters. We develop an efficient algorithm for model inference, based on the collapsed variational Bayes framework with 0th-order Taylor approximation. The merits and efficacy of the proposed nonparametric Bayesian model are demonstrated via challenging applications that concern real-world data clustering and 3D objects recognition.

Stochastic Approximations to the Pitman–Yor Process

In this paper we consider approximations to the popular Pitman–Yor process obtained by truncating the stick-breaking representation. The truncation is determined by a random stopping rule that achieves an almost sure control on the approximation error in total variation distance. We derive the asymptotic distribution of the random truncation point as the approximation error ϵ goes to zero in terms of a polynomially tilted positive stable random variable. The practical usefulness and effectiveness of this theoretical result is demonstrated by devising a sampling algorithm to approximate functionals of the ϵ-version of the Pitman–Yor process.

Sharp Estimates for Geman–Yor Processes and applications to Arithmetic Average Asian options

We prove the existence of the fundamental solution of the degenerate second order partial differential equation related to Geman–Yor stochastic processes, that arise in models for option pricing theory in finance.We then prove pointwise lower and upper bounds for such fundamental solution. Lower bounds are obtained by using repeatedly an invariant Harnack inequality and by solving an associated optimal control problem with quadratic cost. Upper bounds are obtained by the fact that the cost satisfies a specific Hamilton–Jacobi–Bellman equation.

Hierarchical viewpoint discovery from tweets using Bayesian modelling

When users express their stances towards a topic in social media, they might elaborate their viewpoints or reasoning. Oftentimes, viewpoints expressed by different users exhibit a hierarchical structure. Therefore, detecting this kind of hierarchical viewpoints offers a better insight to understand the public opinion. In this paper, we propose a novel Bayesian model for hierarchical viewpoint discovery from tweets. Driven by the motivation that a viewpoint expressed in a tweet can be regarded as a path from the root to a leaf of a hierarchical viewpoint tree, the assignment of the relevant viewpoint topics is assumed to follow two nested Chinese restaurant processes. Moreover, opinions in text are often expressed in un-semantically decomposable multi-terms or phrases, such as ‘economic recession’. Hence, a hierarchical Pitman–Yor process is employed as a prior for modelling the generation of phrases with arbitrary length. Experimental results on two Twitter corpora demonstrate the effectiveness of the proposed Bayesian model for hierarchical viewpoint discovery.

Sequential sampling for CGMY processes via decomposition of their time changes

AbstractWe present a new and easy‐to‐implement sequential sampling method for Carr–Geman–Madan–Yor (CGMY) processes with either finite or infinite variation, exploiting the time change representation of the CGMY model and a decomposition of its time change. We find that the time change can be decomposed into two independent components. While the first component is a finite generalized gamma convolution process whose increments can be sampled by either the exact double CFTP (“coupling from the past”) method or an approximation scheme with high speed and accuracy, the second component can easily be made arbitrarily small in the L1 sense. Simulation results show that the proposed method is advantageous over two existing methods under a model calibrated to historical option price data.

Multi-Armed Bandit for Species Discovery: A Bayesian Nonparametric Approach

ABSTRACTLet (P1, …, PJ) denote J populations of animals from distinct regions. A priori, it is unknown which species are present in each region and what are their corresponding frequencies. Species are shared among populations and each species can be present in more than one region with its frequency varying across populations. In this article, we consider the problem of sequentially sampling these populations to observe the greatest number of different species. We adopt a Bayesian nonparametric approach and endow (P1, …, PJ) with a hierarchical Pitman–Yor process prior. As a consequence of the hierarchical structure, the J unknown discrete probability measures share the same support, that of their common random base measure. Given this prior choice, we propose a sequential rule that, at every time step, given the information available up to that point, selects the population from which to collect the next observation. Rather than picking the population with the highest posterior estimate of producing a new value, the proposed rule includes a Thompson sampling step to better balance the exploration–exploitation trade-off. We also propose an extension of the algorithm to deal with incidence data, where multiple observations are collected in a time period. The performance of the proposed algorithms is assessed through a simulation study and compared to three other strategies. Finally, we compare these algorithms using a dataset of species of trees, collected from different plots in South America. Supplementary materials for this article are available online.

Limiting behaviour of the stationary search cost distribution driven by a generalized gamma process

Consider a list of labeled objects that are organized in a heap. At each time, object $j$ is selected with probability $p_j$ and moved to the top of the heap. This procedure defines a Markov chain on the set of permutations which is referred to in the literature as Move-to-Front rule. The present contribution focuses on the stationary search cost, namely the position of the requested item in the heap when the Markov chain is in equilibrium. We consider the scenario where the number of objects is infinite and the probabilities $p_j$’s are defined as the normalization of the increments of a subordinator. In this setting, we provide an exact formula for the moments of any order of the stationary search cost distribution. We illustrate the new findings in the case of a generalized gamma subordinator and deal with an extension to the two–parameter Poisson–Dirichlet process, also known as Pitman–Yor process.

The Pitman–Yor Process and an Empirical Study of Choice Behavior

This study discusses choice behavior using a voting model in which voters can obtain information from a finite number of previous $r$ voters. Voters vote for a candidate with a probability proportional to the previous vote ratio, which is visible to the voters. We obtain the Pitman sampling formula as the equilibrium distribution of $r$ votes. We present the model as a process of posting on a bulletin board system, 2ch.net, where users can choose one of many threads to create a post. We explore how this choice depends on the last $r$ posts and the distribution of these last $r$ posts across threads. We conclude that the posting process is described by our voting model with analog herders for a small $r$, which might correspond to the time horizon of users' responses.

On the Pitman–Yor process with spike and slab base measure

SummaryFor the most popular discrete nonparametric models, beyond the Dirichlet process, the prior guess at the shape of the data-generating distribution, also known as the base measure, is assumed to be diffuse. Such a specification greatly simplifies the derivation of analytical results, allowing for a straightforward implementation of Bayesian nonparametric inferential procedures. However, in several applied problems the available prior information leads naturally to the incorporation of an atom into the base measure, and then the Dirichlet process is essentially the only tractable choice for the prior. In this paper we fill this gap by considering the Pitman–Yor process with an atom in its base measure. We derive computable expressions for the distribution of the induced random partitions and for the predictive distributions. These findings allow us to devise an effective generalized Pólya urn Gibbs sampler. Applications to density estimation, clustering and curve estimation, with both simulated and real data, serve as an illustration of our results and allow comparisons with existing methodology. In particular, we tackle a functional data analysis problem concerning basal body temperature curves.

Limit theorems associated with the Pitman–Yor process

Abstract The Pitman–Yor process is a random discrete measure. The random weights or masses follow the two-parameter Poisson–Dirichlet distribution with parameters 0 < α < 1, θ > -α. The parameters α and θ correspond to the stable and gamma components, respectively. The distribution of atoms is given by a probability η. In this paper we consider the limit theorems for the Pitman–Yor process and the two-parameter Poisson–Dirichlet distribution. These include the law of large numbers, fluctuations, and moderate or large deviation principles. The limiting procedures involve either α tending to 0 or 1. They arise naturally in genetics and physics such as the asymptotic coalescence time for explosive branching process and the approximation to the generalized random energy model for disordered systems.

A Marginal Sampler for σ-Stable Poisson–Kingman Mixture Models

ABSTRACTWe investigate the class of σ-stable Poisson–Kingman random probability measures (RPMs) in the context of Bayesian nonparametric mixture modeling. This is a large class of discrete RPMs, which encompasses most of the popular discrete RPMs used in Bayesian nonparametrics, such as the Dirichlet process, Pitman–Yor process, the normalized inverse Gaussian process, and the normalized generalized Gamma process. We show how certain sampling properties and marginal characterizations of σ-stable Poisson–Kingman RPMs can be usefully exploited for devising a Markov chain Monte Carlo (MCMC) algorithm for performing posterior inference with a Bayesian nonparametric mixture model. Specifically, we introduce a novel and efficient MCMC sampling scheme in an augmented space that has a small number of auxiliary variables per iteration. We apply our sampling scheme to a density estimation and clustering tasks with unidimensional and multidimensional datasets, and compare it against competing MCMC sampling schemes. Supplementary materials for this article are available online.

Nonparametric Bayesian topic modelling with the hierarchical Pitman–Yor processes

The Dirichlet process and its extension, the Pitman–Yor process, are stochastic processes that take probability distributions as a parameter. These processes can be stacked up to form a hierarchical nonparametric Bayesian model. In this article, we present efficient methods for the use of these processes in this hierarchical context, and apply them to latent variable models for text analytics. In particular, we propose a general framework for designing these Bayesian models, which are called topic models in the computer science community. We then propose a specific nonparametric Bayesian topic model for modelling text from social media. We focus on tweets (posts on Twitter) in this article due to their ease of access. We find that our nonparametric model performs better than existing parametric models in both goodness of fit and real world applications.

Dynamic density estimation with diffusive Dirichlet mixtures

We introduce a new class of nonparametric prior distributions on the space of continuously varying densities, induced by Dirichlet process mixtures which diffuse in time. These select time-indexed random functions without jumps, whose sections are continuous or discrete distributions depending on the choice of kernel. The construction exploits the widely used stick-breaking representation of the Dirichlet process and induces the time dependence by replacing the stick-breaking components with one-dimensional Wright-Fisher diffusions. These features combine appealing properties of the model, inherited from the Wright-Fisher diffusions and the Dirichlet mixture structure, with great flexibility and tractability for posterior computation. The construction can be easily extended to multi-parameter GEM marginal states, which include, for example, the Pitman--Yor process. A full inferential strategy is detailed and illustrated on simulated and real data.

Omega diffusion risk model with surplus-dependent tax and capital injections

In this paper, we propose and study an Omega risk model with a constant bankruptcy function, surplus-dependent tax payments and capital injections in a time-homogeneous diffusion setting. The surplus value process is both refracted (paying tax) at its running maximum and reflected (injecting capital) at a lower constant boundary. The new model incorporates practical features from the Omega risk model (Albrecher et al., 2011), the risk model with tax (Albrecher and Hipp, 2007), and the risk model with capital injections (Albrecher and Ivanovs, 2014). The study of this new risk model is closely related to the Azéma–Yor process, which is a process refracted by its running maximum. We explicitly characterize the Laplace transform of the occupation time of an Azéma–Yor process below a constant level until the first passage time of another Azéma–Yor process or until an independent exponential time. We also consider the case when the process has a lower reflecting boundary. This result unifies and extends recent results of Li and Zhou (2013) and Zhang (2015). We explicitly characterize the Laplace transform of the time of bankruptcy in the Omega risk model with tax and capital injections up to eigen-functions, and determine the expected present value of tax payments until default. We also discuss a further extension to occupation functionals through stochastic time-change, which handles the case of a non-constant bankruptcy function. Finally we present examples using a Brownian motion with drift, and discuss the pricing of quantile options written on the Azéma–Yor process.

Value-at-risk estimation with the Carr–Geman–Madan–Yor process: an empirical study on foreign exchange rates

To discover whether the Carr-Geman-Madan-Yor (CGMY) distribution, a Lévy process that can have an infinite number of jumps in a finite time interval, is appropriate for describing the fat tails of the distribution of foreign exchange (FX) rate returns, we investigate its performance in estimating the risk of FX rates. We estimate valu-at-risk (VaR), one of the most popular concepts in the area of risk management, for FX rate returns. To enhance the robustness of the estimation results, we use other VaR models, such as historical simulation, the generalized autoregressive conditional heteroscedasticity (GARCH) model and conditional extreme value theory (EVT). Our four methodologies are applied to three major foreign exchange rates (EUR/JPY, EUR/USD and USD/JPY) by using a different size estimation windows for each model to forecast one-day-ahead VaR. We propose a new procedure to find the best size of estimation window for each VaR model, with which we compare the respective VaR estimates. The conditional EVT and CGMY distributions provide superior forecasting performance for all three rates's daily returns, respectively. Furthermore, the performance of the other methods can be significantly improved by adjusting the size of the estimation window.

Statistical models for language representation

ONTARE. REVISTA DE INVESTIGACIÓN DE LA FACULTAD DE INGENIERÍA
 This paper discuses several models for the computational representation of language. First, some n-gram models that are based on Markov models are introduced. Second, a family of models known as the exponential models is taken into account. This family in particular allows the incorporation of several features to model. Third, a recent current of research, the probabilistic Bayesian approach, is discussed. In this kind of models, language is modeled as a probabilistic distribution. Several distributions and probabilistic processes, such as the Dirichlet distribution and the Pitman- Yor process, are used to approximate the linguistic phenomena. Finally, the problem of sparseness of the language and its common solution known as smoothing is discussed.

Beta-product dependent Pitman–Yor processes for Bayesian inference

Multiple time series data may exhibit clustering over time and the clustering effect may change across different series. This paper is motivated by the Bayesian non-parametric modelling of the dependence between clustering effects in multiple time series analysis. We follow a Dirichlet process mixture approach and define a new class of multivariate dependent Pitman–Yor processes (DPY). The proposed DPY are represented in terms of vectors of stick-breaking processes which determine dependent clustering structures in the time series. We follow a hierarchical specification of the DPY base measure to account for various degrees of information pooling across the series. We discuss some theoretical properties of the DPY and use them to define Bayesian non-parametric repeated measurement and vector autoregressive models. We provide efficient Monte Carlo Markov Chain algorithms for posterior computation of the proposed models and illustrate the effectiveness of the method with a simulation study and an application to the United States and the European Union business cycle.

Autonomous tactile perception: A combined improved sensing and Bayesian nonparametric approach

In recent years, autonomous robots have increasingly been deployed in unknown environments and required to manipulate or categorize unknown objects. In order to cope with these unfamiliar situations, improvements must be made both in sensing technologies and in the capability to autonomously train perception models. In this paper, we explore this problem in the context of tactile surface identification and categorization. Using a highly-discriminant tactile probe based upon large bandwidth, triple axis accelerometer that is sensitive to surface texture and material properties, we demonstrate that unsupervised learning for surface identification with this tactile probe is feasible. To this end, we derived a Bayesian nonparametric approach based on Pitman–Yor processes to model power-law distributions, an extension of our previous work using Dirichlet processes Dallaire et al. (2011). When tested against a large collection of surfaces and without providing the actual number of surfaces, the tactile probe combined with our proposed approach demonstrated near-perfect recognition in many cases and achieved perfect recognition given the right conditions. We consider that our combined improvements demonstrate the feasibility of effective autonomous tactile perception systems.

A latent variable Gaussian process model with Pitman–Yor process priors for multiclass classification

Gaussian processes (GPs) constitute one of the most important Bayesian machine learning approaches. Several researchers have considered postulating mixtures of Gaussian processes as a means of dealing with non-stationary covariance functions, discontinuities, multi-modality, and overlapping output signals. In existing works, mixtures of Gaussian processes are based on the introduction of a gating function defined over the space of model input variables. This way, each postulated mixture component Gaussian process is effectively restricted in a limited subset of the input space. Additionally, the applicability of these models is limited to regression tasks. In this paper, for the first time in the literature, we devise a Gaussian process mixture model especially suitable for multiclass classification applications: We consider a GP classification scheme the prior distribution of which is a fully generative nonparametric Bayesian model with power-law behavior, generating Gaussian processes over the whole input space of the learned task. We provide an efficient algorithm for model inference, based on the variational Bayesian framework, and exhibit its efficacy using benchmark and real-world classification datasets.