Nonparametric Bayesian Inference Research Articles

Abstract TP61: Automated Labeling of Cerebral Vasculature on MR Angiography Using Nonparametric Bayesian Inference

Background: An automated, unbiased method to accurately label cerebrovascular territories would greatly advance our ability to assess individual stroke patients as well as study large databases. Previous attempts have failed due to the wide variation in normal vascular topography. We test the hypothesis that a nonparametric probabilistic model that learns the configurational characteristics of vascular territories will better annotate the cerebrovasculature. Methods: In the George Mason Brain Vasculature database, we identified patients with MRA reconstructions segmented into seven major regions (left and right MCA, PCA, and ACA and Circle of Willis). We then augment these labels by manually segmenting the MCA territory into an additional eight regions. We then divide the database into training and validation cohorts to assess the algorithm. Results: Among 54 patients that met the inclusion criteria, 39 reconstructions were used as training input to the model among the 61 digital reconstructions of human brain arterial structures available. The model was then validated on an independent cohort of 15 patients. The algorithm was found to be 94+-5.2% accurate in annotating the vascular segments. Conclusions: Kernel density estimation, used in conjunction with a Bayesian inference-based algorithm, can accurately label cerebral vascular territories using spatial and radial features. This process can provide a framework for further vascular segmentation and analysis of artery occlusion in stroke patients.

String and membrane Gaussian processes

In this paper we introduce a novel framework for making exact nonparametric Bayesian inference on latent functions that is particularly suitable for Big Data tasks. Firstly, we introduce a class of stochastic processes we refer to as string Gaussian processes (string GPs which are not to be mistaken for Gaussian processes operating on text). We construct string GPs so that their finite-dimensional marginals exhibit suitable local conditional independence structures, which allow for scalable, distributed, and flexible nonparametric Bayesian inference, without resorting to approximations, and while ensuring some mild global regularity constraints. Furthermore, string GP priors naturally cope with heterogeneous input data, and the gradient of the learned latent function is readily available for explanatory analysis. Secondly, we provide some theoretical results relating our approach to the standard GP paradigm. In particular, we prove that some string GPs are Gaussian processes, which provides a complementary global perspective on our framework. Finally, we derive a scalable and distributed MCMC scheme for supervised learning tasks under string GP priors. The proposed MCMC scheme has computational time complexity O(N) and memory requirement O(dN), where N is the data size and d the dimension of the input space. We illustrate the efficacy of the proposed approach on several synthetic and real-world data sets, including a data set with 6 millions input points and 8 attributes.

Posterior contraction rates for deconvolution of Dirichlet-Laplace mixtures

We study nonparametric Bayesian inference with location mixtures of the Laplace density and a Dirichlet process prior on the mixing distribution. We derive a contraction rate of the corresponding posterior distribution, both for the mixing distribution relative to the Wasserstein metric and for the mixed density relative to the Hellinger and $L_q$ metrics.

Bayesian Nonparametric Inference of Population Size Changes from Sequential Genealogies.

Sophisticated inferential tools coupled with the coalescent model have recently emerged for estimating past population sizes from genomic data. Recent methods that model recombination require small sample sizes, make constraining assumptions about population size changes, and do not report measures of uncertainty for estimates. Here, we develop a Gaussian process-based Bayesian nonparametric method coupled with a sequentially Markov coalescent model that allows accurate inference of population sizes over time from a set of genealogies. In contrast to current methods, our approach considers a broad class of recombination events, including those that do not change local genealogies. We show that our method outperforms recent likelihood-based methods that rely on discretization of the parameter space. We illustrate the application of our method to multiple demographic histories, including population bottlenecks and exponential growth. In simulation, our Bayesian approach produces point estimates four times more accurate than maximum-likelihood estimation (based on the sum of absolute differences between the truth and the estimated values). Further, our method’s credible intervals for population size as a function of time cover 90% of true values across multiple demographic scenarios, enabling formal hypothesis testing about population size differences over time. Using genealogies estimated with ARGweaver, we apply our method to European and Yoruban samples from the 1000 Genomes Project and confirm key known aspects of population size history over the past 150,000 years.

A Brief Ode to the Empirical Likelihood Concept

robust likelihood-type techniques. Towards this end, the modern biostatistical literature has shifted focus towards robust and efficient nonparametric likelihood methods [2]. The empirical likelihood (EL) methodology employs the likelihood concept in a distribution-free manner, approximating optimal parametric likelihood-based procedures. Since EL techniques and parametric likelihood methods are closely related concepts, one may apply corresponding EL functions to replace their parametric likelihood counterparts in known and well developed parametric procedures, e.g. constructing novel nonparametric Bayesian inference [3] and the confidence interval estimation [4]. This provides the impetus for an impressive expansion in the number of EL developments based on combinations of likelihoods of different types, e.g., when incomplete data should be analyzed, nonparametric likelihood ratio techniques can be combined with parametric likelihoods [5]. The classical EL methodology, which is a distribution functionbased approach, has been shown to have attractive properties for testing hypotheses regarding parameters (e.g. moments) of distributions [6]. In practice, statisticians commonly face a variety of distribution-free comparisons and/or evaluations over all distribution functions of complete and incomplete data subject to different types of measurement errors. In these frameworks, the density-based EL methodology is shown to be very efficient [710]. According to the Neyman-Pearson lemma, the most powerful test statistics have structures that are related to density-based likelihood ratios. The density-based EL method can be easily and satisfactorily applied to construct highly efficient test procedures, approximating non-parametrically most powerful NeymanPearson test-rules, given aims of clinical studies. Similarly to the parametric likelihood concept, the EL methodology provides relatively simple strategies to construct powerful statistical tests that can be applied in various complex biostatistical studies. The extreme generality of EL methods and their wide ranges of usefulness partly result from the simple derivation of the EL statistics as components of composite parametric and nonparametric likelihood based systems, efficiently attending to any observed data and relevant information. The EL based methods are employed in much of modern biostatistical practice.

HiHMM: Bayesian non-parametric joint inference of chromatin state maps.

Motivation: Genome-wide mapping of chromatin states is essential for defining regulatory elements and inferring their activities in eukaryotic genomes. A number of hidden Markov model (HMM)-based methods have been developed to infer chromatin state maps from genome-wide histone modification data for an individual genome. To perform a principled comparison of evolutionarily distant epigenomes, we must consider species-specific biases such as differences in genome size, strength of signal enrichment and co-occurrence patterns of histone modifications.Results: Here, we present a new Bayesian non-parametric method called hierarchically linked infinite HMM (hiHMM) to jointly infer chromatin state maps in multiple genomes (different species, cell types and developmental stages) using genome-wide histone modification data. This flexible framework provides a new way to learn a consistent definition of chromatin states across multiple genomes, thus facilitating a direct comparison among them. We demonstrate the utility of this method using synthetic data as well as multiple modENCODE ChIP-seq datasets.Conclusion: The hierarchical and Bayesian non-parametric formulation in our approach is an important extension to the current set of methodologies for comparative chromatin landscape analysis.Availability and implementation: Source codes are available at https://github.com/kasohn/hiHMM. Chromatin data are available at http://encode-x.med.harvard.edu/data_sets/chromatin/.Contact: peter_park@harvard.edu or juhan@snu.ac.krSupplementary information: Supplementary data are available at Bioinformatics online.

GreenLocs

Understanding indoor mobility patterns of people is important in applications such as targeted advertisement, microclimate control, and delivery of anticipatory notifications. In this article, we devise GreenLocs, a nonparametric, profiling-free, yet lightweight and energy-efficient inference framework, to identify recurring and new places that mobile users visit indoor. Combining WiFi scans and accelerometer readings, GreenLocs can accurately decide a new place and a revisited place with just a few radio signal strength (RSS) samples. GreenLocs consists of three major building blocks, namely, missing data handling algorithms, a nonparametric Bayesian inference model, and a stopping rule, which significantly increases the energy efficiency of the system. GreenLocs is shown to be robust to signal variations and missing data through experimental evaluations using traces collected from mobile phones of different brands/models.

Are Gibbs-Type Priors the Most Natural Generalization of the Dirichlet Process?

Discrete random probability measures and the exchangeable random partitions they induce are key tools for addressing a variety of estimation and prediction problems in Bayesian inference. Here we focus on the family of Gibbs-type priors, a recent elegant generalization of the Dirichlet and the Pitman-Yor process priors. These random probability measures share properties that are appealing both from a theoretical and an applied point of view: (i) they admit an intuitive predictive characterization justifying their use in terms of a precise assumption on the learning mechanism; (ii) they stand out in terms of mathematical tractability; (iii) they include several interesting special cases besides the Dirichlet and the Pitman-Yor processes. The goal of our paper is to provide a systematic and unified treatment of Gibbs-type priors and highlight their implications for Bayesian nonparametric inference. We deal with their distributional properties, the resulting estimators, frequentist asymptotic validation and the construction of time-dependent versions. Applications, mainly concerning mixture models and species sampling, serve to convey the main ideas. The intuition inherent to this class of priors and the neat results they lead to make one wonder whether it actually represents the most natural generalization of the Dirichlet process.

Analyzing Single-Molecule Time Series via Nonparametric Bayesian Inference

The ability to measure the properties of proteins at the single-molecule level offers an unparalleled glimpse into biological systems at the molecular scale. The interpretation of single-molecule time series has often been rooted in statistical mechanics and the theory of Markov processes. While existing analysis methods have been useful, they are not without significant limitations including problems of model selection and parameter nonidentifiability. To address these challenges, we introduce the use of nonparametric Bayesian inference for the analysis of single-molecule time series. These methods provide a flexible way to extract structure from data instead of assuming models beforehand. We demonstrate these methods with applications to several diverse settings in single-molecule biophysics. This approach provides a well-constrained and rigorously grounded method for determining the number of biophysical states underlying single-molecule data.

그리드 단체 위의 디리슐레 분포에서 마르코프 연쇄 몬테 칼로 표집

With the recent machine learning paradigm of using nonparametric Bayesian statistics and statistical inference based on random sampling, the Dirichlet distribution finds many uses in a variety of graphical models. It is a multivariate generalization of the gamma distribution and is defined on a continuous (K-1)-simplex. This paper presents a sampling method for a Dirichlet distribution for the problem of dividing an integer X into a sequence of K integers which sum to X. The target samples in our problem are all positive integer vectors when multiplied by a given X. They must be sampled from the correspondingly gridded simplex. In this paper we develop a Markov Chain Monte Carlo (MCMC) proposal distribution for the neighborhood grid points on the simplex and then present the complete algorithm based on the Metropolis-Hastings algorithm. The proposed algorithm can be used for the Markov model, HMM, and Semi-Markov model for accurate state-duration modeling. It can also be used for the Gamma-Dirichlet HMM to model q the global-local duration distributions.

Large deviation principles for the Ewens-Pitman sampling model

Let $M_{l,n}$ be the number of blocks with frequency $l$ in the exchangeable random partition induced by a sample of size $n$ from the Ewens-Pitman sampling model. In this paper we show that, as $n$ tends to infinity, $n^{-1}M_{l,n}$ satisfies a large deviation principle and we characterize the corresponding rate function. A conditional counterpart of this large deviation principle is also presented. Specifically, given an initial observed sample of size $n$ from the Ewens-Pitman sampling model, we consider an additional unobserved sample of size $m$ thus giving rise to an enlarged sample of size $n+m$. Then, for any fixed $n$ and as $m$ tends to infinity, we establish a large deviation principle for the conditional number of blocks with frequency $l$ in the enlarged sample, given the initial sample. Interestingly this conditional large deviation principle coincides with the large deviation principle for $M_{l,n}$, namely there is no long lasting impact of the given initial sample to the large deviations. Potential applications of our conditional large deviation principle are thoroughly discussed in the context of Bayesian nonparametric inference for species sampling problems.

Marginally specified priors for non-parametric Bayesian estimation

Prior specification for nonparametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. Realistically, a statistician is unlikely to have informed opinions about all aspects of such a parameter, but may have real information about functionals of the parameter, such the population mean or variance. This article proposes a new framework for nonparametric Bayes inference in which the prior distribution for a possibly infinite-dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a nonparametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard nonparametric prior distributions in common use, and inherit the large support of the standard priors upon which they are based. Additionally, posterior approximations under these informative priors can generally be made via minor adjustments to existing Markov chain approximation algorithms for standard nonparametric prior distributions. We illustrate the use of such priors in the context of multivariate density estimation using Dirichlet process mixture models, and in the modeling of high-dimensional sparse contingency tables.

Sensing-Transmission Edifice Using Bayesian Nonparametric Traffic Clustering in Cognitive Radio Networks

In cognitive radio networks, the main objective of spectrum sensing is to exploit spectrum holes left by the primary users (PUs). Different PUs' traffic patterns might provide different opportunities for second user (SU) spectrum access. In this paper, we identify the PUs' traffic patterns and then maximize SU transmission accordingly. First a theoretical framework is developed to cluster PU traffic patterns based on a Bayesian nonparametric inference model, in which the number of traffic types is unknown. Second, in order to exploit the spectrum holes, we study a sensing-transmission structure to optimize the SU transmission strategy. Specifically, we exploit the short and long transmission opportunities based on the PU traffic pattern and channel idle time distribution. Finally, we propose a threshold-based sensing-transmission method that optimizes the SU utility, while protecting PU transmissions. Both sensing and transmission errors are considered for perfect sensing with/without acknowledgement-based transmission and imperfect sensing, respectively. From the simulation results, we show that the proposed technique outperforms the nonparametric mean shift clustering algorithm. Furthermore, we utilize these clustering results to optimize the SU's transmission strategy with perfect and imperfect sensing. We compare our proposed technique with the probabilistic sensing-transmission structure and show the performance gain in terms of throughput.

Bayesian nonparametric inference for shared species richness in multiple populations

We introduce the branching Ewens–Pitman sampling model for dependent species sequences. The model defines random probability measures having marginally two-parameter Poisson–Dirichlet process distributions. These random measures are associated with the nodes of a binary tree which describes the strength of dependence of the resulting random partitions. We discuss Bayesian analysis under the introduced model and provide algorithms for posterior inference. The model is applied to evaluate similarities across microbial populations in the human esophagus.

Posterior analysis of rare variants in Gibbs-type species sampling models

Species sampling problems have a long history in ecological and biological studies and a number of statistical issues, including the evaluation of species richness, are still to be addressed. In this paper, motivated by Bayesian nonparametric inference for species sampling problems, we consider the practically important and technically challenging issue of developing a comprehensive posterior analysis of the so-called rare variants, namely those species with frequency less than or equal to a given abundance threshold. In particular, by adopting a Gibbs-type prior, we provide an explicit expression for the posterior joint distribution of the frequency counts of the rare variants, and we investigate some of its statistical properties. The proposed results are illustrated by means of two novel applications to a benchmark genomic dataset.

Discussion of “On simulation and properties of the stable law” by L. Devroye and L. James

We contribute to the discussion of the paper by Devroye and James, by reviewing some of the most meaningful results that relate the unilateral stable distribution with the asymptotic behavior of the so-called Ewens-Pitman sampling model. Our focus is then on how these results have been exploited in the context of Bayesian nonparametric inference for species sampling problems.

Goodness-of-fit tests based on the distance between the Dirichlet process and its base measure

The Dirichlet process is a fundamental tool in studying Bayesian nonparametric inference. The Dirichlet process has several sum representations, where each one of these representations highlights some aspects of this important process. In this paper, we use the sum representations of the Dirichlet process to derive explicit expressions that are used to calculate Kolmogorov, Lévy, and Cramér–von Mises distances between the Dirichlet process and its base measure. The derived expressions of the distance are used to select a proper value for the concentration parameter of the Dirichlet process. These tools are also used in a goodness-of-fit test. Illustrative examples and simulation results are included.

Marginally specified priors for non-parametric Bayesian estimation.

Prior specification for non-parametric Bayesian inference involves the difficult task of quantifying prior knowledge about a parameter of high, often infinite, dimension. A statistician is unlikely to have informed opinions about all aspects of such a parameter but will have real information about functionals of the parameter, such as the population mean or variance. The paper proposes a new framework for non-parametric Bayes inference in which the prior distribution for a possibly infinite dimensional parameter is decomposed into two parts: an informative prior on a finite set of functionals, and a non-parametric conditional prior for the parameter given the functionals. Such priors can be easily constructed from standard non-parametric prior distributions in common use and inherit the large support of the standard priors on which they are based. Additionally, posterior approximations under these informative priors can generally be made via minor adjustments to existing Markov chain approximation algorithms for standard non-parametric prior distributions. We illustrate the use of such priors in the context of multivariate density estimation using Dirichlet process mixture models, and in the modelling of high dimensional sparse contingency tables.

Asymptotics for the number of blocks in a conditional Ewens-Pitman sampling model

The study of random partitions has been an active research area in probability over the last twenty years. A quantity that has attracted a lot of attention is the number of blocks in the random partition. Depending on the area of applications this quantity could represent the number of species in a sample from a population of individuals or he number of cycles in a random permutation, etc. In the context of Bayesian nonparametric inference such a quantity is associated with the exchangeable random partition induced by sampling from certain prior models, for instance the Dirichlet process and the two parameter Poisson-Dirichlet process. In this paper we generalize some existing asymptotic results from this prior setting to the so-called posterior, or conditional, setting. Specifically, given an initial sample from a two parameter Poisson-Dirichlet process, we establish conditional fluctuation limits and conditional large deviation principles for the number of blocks generated by a large additional sample.

On Some Optimal Bayesian Nonparametric Rules for Estimating Distribution Functions

In this paper, we present a novel approach to estimating distribution functions, which combines ideas from Bayesian nonparametric inference, decision theory and robustness. Given a sample from a Dirichlet process on the space (𝒳, A), with parameter η in a class of measures, the sampling distribution function is estimated according to some optimality criteria (mainly minimax and regret), when a quadratic loss function is assumed. Estimates are then compared in two examples: one with simulated data and one with gas escapes data in a city network.