Bayesian Nonparametric Model Research Articles

Unsupervised Grouped Axial Data Modeling via Hierarchical Bayesian Nonparametric Models With Watson Distributions.

This paper aims at proposing an unsupervised hierarchical nonparametric Bayesian framework for modeling axial data (i.e., observations are axes of direction) that can be partitioned into multiple groups, where each observation within a group is sampled from a mixture of Watson distributions with an infinite number of components that are allowed to be shared across different groups. First, we propose a hierarchical nonparametric Bayesian model for modeling grouped axial data based on the hierarchical Pitman-Yor process mixture model of Watson distributions. Then, we demonstrate that by setting the discount parameters of the proposed model to 0, another hierarchical nonparametric Bayesian model based on hierarchical Dirichlet process can be derived for modeling axial data. To learn the proposed models, we systematically develop a closed-form optimization algorithm based on the collapsed variational Bayes (CVB) inference. Furthermore, to ensure the convergence of the proposed learning algorithm, an annealing mechanism is introduced to the framework of CVB inference, leading to an averaged collapsed variational Bayes inference strategy. The merits of the proposed models for modeling grouped axial data are demonstrated through experiments on both synthetic data and real-world applications involving gene expression data clustering and depth image analysis.

Semi-Supervised Non-Parametric Bayesian Modelling of Spatial Proteomics.

Understanding sub-cellular protein localisation is an essential component in the analysis of context specific protein function. Recent advances in quantitative mass-spectrometry (MS) have led to high resolution mapping of thousands of proteins to sub-cellular locations within the cell. Novel modelling considerations to capture the complex nature of these data are thus necessary. We approach analysis of spatial proteomics data in a non-parametric Bayesian framework, using K-component mixtures of Gaussian process regression models. The Gaussian process regression model accounts for correlation structure within a sub-cellular niche, with each mixture component capturing the distinct correlation structure observed within each niche. The availability of marker proteins (i.e. proteins with a priori known labelled locations) motivates a semi-supervised learning approach to inform the Gaussian process hyperparameters. We moreover provide an efficient Hamiltonian-within-Gibbs sampler for our model. Furthermore, we reduce the computational burden associated with inversion of covariance matrices by exploiting the structure in the covariance matrix. A tensor decomposition of our covariance matrices allows extended Trench and Durbin algorithms to be applied to reduce the computational complexity of inversion and hence accelerate computation. We provide detailed case-studies on Drosophila embryos and mouse pluripotent embryonic stem cells to illustrate the benefit of semi-supervised functional Bayesian modelling of the data.

Nonparametric Bayesian modeling for hazard functions in spatial survival data

Survival analysis is a collection of statistical procedures for data analysis where the outcome variable of interest is time until an event occurs. The Cox proportional hazard is the most widely used in survival analysis as it takes advantage that it accounts for the proportionate risk of covariates without estimating baseline hazards. However, it is necessary to obtain the baseline hazard function as well as regression parameters to describe the exact hazards of a study population. When data is collected by clinical sites or geographical regions, we should reflect the spatial structures into the model. In this paper we adopted a nonparametric Bayesian hierarchical model to estimate the hazard function in spatial survival data. We assume the cumulative baseline hazard function as a monotone step function, considering flexible priors called stick-breaking priors. We provide modeling for the hazard function that can describe spatial structure, and apply it to survival data which is observed in 24 administrative districts in the northwest of England.

Comorbidity clusters associated with newly treated type 2 diabetes mellitus: a Bayesian nonparametric analysis

Type 2 diabetes mellitus (T2DM) is associated with the development of chronic comorbidities, which can lead to high drug utilization and adverse events. We aimed to identify common comorbidity clusters and explore the progression over time in newly treated T2DM patients. The IQVIA Medical Research Data incorporating data from THIN, a Cegedim database of anonymized electronic health records, was used to identify all patients with a first-ever prescription for a non-insulin antidiabetic drug (NIAD) between January 2006 and December 2019. We selected 58 chronic comorbidities of interest and used Bayesian nonparametric models to identify disease clusters and model their progression over time. Among the 175,383 eligible T2DM patients, we identified the 20 most frequent comorbidity clusters, which were comprised of 14 latent features (LFs). Each LF was associated with a primary disease (e.g., 98% of patients in cluster 2, characterized by LF2, had congestive heart failure [CHF]). The presence of certain LFs increased the probability of having another LF active. For example, LF2 (CHF) frequently appeared with LFs related to chronic kidney disease (CKD). Over time, the clusters associated with cardiovascular diseases, such as CHF, progressed rapidly. Moreover, the onset of certain diseases led to further complications. Our models identified established T2DM complications and previously unknown connections, thus, highlighting the potential for Bayesian nonparametric models to characterize complex comorbidity patterns.

Bayesian Spatial Blind Source Separation via the Thresholded Gaussian Process

Blind source separation (BSS) aims to separate latent source signals from their mixtures. For spatially dependent signals in high-dimensional and large-scale data, such as neuroimaging, most existing BSS methods do not take into account the spatial dependence and the sparsity of the latent source signals. To address these major limitations, we propose a Bayesian spatial blind source separation (BSP-BSS) approach for neuroimaging data analysis. We assume the expectation of the observed images as a linear mixture of multiple sparse and piece-wise smooth latent source signals, for which we construct a new class of Bayesian nonparametric prior models by thresholding Gaussian processes. We assign the vMF priors to mixing coefficients in the model. Under some regularity conditions, we show that the proposed method has several desirable theoretical properties including the large support for the priors, the consistency of joint posterior distribution of the latent source intensity functions and the mixing coefficients, and the selection consistency on the number of latent sources. We use extensive simulation studies and an analysis of the resting-state fMRI data in the Autism Brain Imaging Data Exchange (ABIDE) study to demonstrate that BSP-BSS outperforms the existing method for separating latent brain networks and detecting activated brain activation in the latent sources. Supplementary materials for this article are available online.

A Bayesian nonparametric method for detecting rapid changes in disease transmission

Whether an outbreak of infectious disease is likely to grow or dissipate is determined through the time-varying reproduction number, Rt. Real-time or retrospective identification of changes in Rt following the imposition or relaxation of interventions can thus contribute important evidence about disease transmission dynamics which can inform policymaking. Here, we present a method for estimating shifts in Rt within a renewal model framework. Our method, which we call EpiCluster, is a Bayesian nonparametric model based on the Pitman–Yor process. We assume that Rt is piecewise-constant, and the incidence data and priors determine when or whether Rt should change and how many times it should do so throughout the series. We also introduce a prior which induces sparsity over the number of changepoints. Being Bayesian, our approach yields a measure of uncertainty in Rt and its changepoints. EpiCluster is fast, straightforward to use, and we demonstrate that it provides automated detection of rapid changes in transmission, either in real-time or retrospectively, for synthetic data series where the Rt profile is known. We illustrate the practical utility of our method by fitting it to case data of outbreaks of COVID-19 in Australia and Hong Kong, where it finds changepoints coinciding with the imposition of non-pharmaceutical interventions. Bayesian nonparametric methods, such as ours, allow the volume and complexity of the data to dictate the number of parameters required to approximate the process and should find wide application in epidemiology. This manuscript was submitted as part of a theme issue on “Modelling COVID-19 and Preparedness for Future Pandemics”.

Bayesian Tree-Structured Two-Level Clustering for Nested Data Analysis

Data integration plays a crucial role in the era of big data. The nested data are a combined set of observations from multiple sources and exhibit heterogeneity both at the source level and at the observational level. The complex nature makes it challenging to reasonably visualize and jointly analyze the nested data. In this article, we present a nonparametric Bayesian model to implement the tree-structured two-level clustering for nested data analysis. The two-level clustering is used to tease out the heterogeneity existing in the sources and observations, while a tree-structured prior is employed to model the latent hierarchy for clusters at the observational level. The proposed Bayesian model is flexible as it does not require an exact specification of cluster numbers or tree width/depth, and it can automatically learn the underlying tree structures among clusters of observations, thus, offering an insightful visualization of the nested data. We further provide a rigorous posterior sampling scheme via the partially collapsed Gibbs sampler and show the performance of the proposed method using simulation studies. Finally, the applications to two different types of nested data (multi-source image data and multi-subject single-cell expression data) demonstrate the advantages of the proposed Bayesian method. Supplementary materials for this article are available online.

Modeling and forecasting of nonlinear nonstationary processes based on the Bayesian structural time series

The article describes an approach to modelling and forecasting non-linear non-stationary time series for various purposes using Bayesian structural time series. The concepts of non-linearity and non-stationarity, as well as methods for processing non-linearity’sand non-stationarity in the construction of forecasting models are considered. The features of the Bayesian approach in the processing of nonlinearities and nonstationaryare presented. An approach to the construction of probabilistic-statistical models based on Bayesian structural models of time series has been studied. Parametric and non-parametric methods for forecasting non-linear and non-stationary time series are considered. Parametric methods include methods: classical autoregressive models, neural networks, models of support vector machines, hidden Markov models. Non-parametric methods include methods: state-space models, functional decomposition models, Bayesian non-parametric models. One of the types of non-parametric models isBayesian structural time series. The main features of constructing structural time series are considered. Models of structural time series are presented. The process of learning the Bayesianstructural model of time series is described. Training is performed in four stages: setting the structure of the model and a priori probabilities; applying a Kalman filter to update state estimates based on observed data;application of the “spike-and-slab”method to select variables in a structural model; Bayesian averaging to combine the results to make a prediction. An algorithm for constructing a Bayesian structural time seriesmodel is presented. Various components of the BSTS model are considered andanalysed, with the help of which the structures of alternative predictive models are formed. As an example of the application of Bayesian structural time series, the problem of predicting Amazon stock prices is considered. The base dataset is amzn_share. After loading, the structure and data types were analysed, and missing values were processed. The data are characterized by irregular registration of observations, which leads to a large number of missing values and “masking” possible seasonal fluctuations. This makes the task of forecasting rather difficult. To restore gaps in the amzn_sharetime series, the linear interpolation method was used. Using a set of statistical tests (ADF, KPSS, PP), the series was tested for stationarity. The data set is divided into two parts: training and testing. The fitting of structural models of time series was performed using the Kalman filterand the Monte Carlo method according to the Markov chain scheme. To estimate and simultaneously regularize the regression coefficients, the spike-and-slab method was applied. The quality of predictive models was assessed.

Key Population Size Estimation to Guide HIV Epidemic Responses in Nigeria: Bayesian Analysis of 3-Source Capture-Recapture Data.

Nigeria has the fourth largest burden of HIV globally. Key populations, including female sex workers, men who have sex with men, and people who inject drugs, are more vulnerable to HIV than the general population due to stigmatized and criminalized behaviors. Reliable key population size estimates are needed to guide HIV epidemic response efforts. The objective of our study was to use empirical methods for sampling and analysis to improve the quality of population size estimates of female sex workers, men who have sex with men, and people who inject drugs in 7 states (Akwa Ibom, Benue, Cross River, Lagos, Nasarawa, Rivers, and the Federal Capital Territory) of Nigeria for program planning and to demonstrate improved statistical estimation methods. From October to December 2018, we used 3-source capture-recapture to produce population size estimates in 7 states in Nigeria. Hotspots were mapped before 3-source capture-recapture started. We sampled female sex workers, men who have sex with men, and people who inject drugs during 3 independent captures about one week apart. During hotspot encounters, key population members were offered inexpensive, memorable objects unique to each capture round. In subsequent rounds, key population members were offered an object and asked to identify objects received during previous rounds (if any). Correct responses were tallied and recorded on tablets. Data were aggregated by key population and state for analysis. Median population size estimates were derived using Bayesian nonparametric latent-class models with 80% highest density intervals. Overall, we sampled approximately 310,000 persons at 9015 hotspots during 3 independent captures. Population size estimates for female sex workers ranged from 14,500 to 64,300; population size estimates for men who have sex with men ranged from 3200 to 41,400; and population size estimates for people who inject drugs ranged from 3400 to 30,400. This was the first implementation of these 3-source capture-recapture methods in Nigeria. Our population size estimates were larger than previously documented for each key population in all states. The Bayesian models account for factors, such as social visibility, that influence heterogeneous capture probabilities, resulting in more reliable population size estimates. The larger population size estimates suggest a need for programmatic scale-up to reach these populations, which are at highest risk for HIV.

A non-parametric Bayesian joint model for latent individual molecular profiles and survival in oncology.

The development of prognostic molecular signatures considering the inter-patient heterogeneity is a key challenge for the precision medicine. We propose a joint model of this heterogeneity and the patient survival, assuming that tumor expression results from a mixture of a subset of independent signatures. We deconvolute the omics data using a non-parametric independent component analysis with a double sparseness structure for the source and the weight matrices, corresponding to the gene-component and individual-component associations, respectively. In a simulation study, our approach identified the correct number of components and reconstructed with high accuracy the weight ([Formula: see text]0.85) and the source ([Formula: see text]0.75) matrices sparseness. The selection rate of components with high-to-moderate prognostic impacts was close to 95%, while the weak impacts were selected with a frequency close to the observed false positive rate ([Formula: see text]25%). When applied to the expression of 1063 genes from 614 breast cancer patients, our model identified 15 components, including six associated to patient survival, and related to three known prognostic pathways in early breast cancer (i.e. immune system, proliferation, and stromal invasion). The proposed algorithm provides a new insight into the individual molecular heterogeneity that is associated with patient prognosis to better understand the complex tumor mechanisms.

Nonparametric Bayesian inference for reversible multidimensional diffusions

We study nonparametric Bayesian models for reversible multidimensional diffusions with periodic drift. For continuous observation paths, reversibility is exploited to prove a general posterior contraction rate theorem for the drift gradient vector field under approximation-theoretic conditions on the induced prior for the invariant measure. The general theorem is applied to Gaussian priors and p-exponential priors, which are shown to converge to the truth at the optimal nonparametric rate over Sobolev smoothness classes in any dimension.

Application of Bayesian Semi-Parametric and Hierarchical Models for Analyzing Dispersed Traffic Barriers Crash Data

Introduction: Despite the traffic barriers effectiveness in reduction of the severity of run-off road crashes, the severity of barrier crashes still accounts for a significant fraction of road fatalities. Although extensive research has already been conducted in studying traffic barrier crashes, those studies mostly either consider the severity or frequency of crashes. Here, the equivalent property damage only (EPDO) was used to account for both aspects of crashes. While modeling EPDO crashes, there are challenges associated with that type of dataset including its sparse distribution, and the presence of heterogeneity in the dataset due to aggregation of various crash types. Methods: Ignoring the sparse nature of the data might result in biased or even erroneous results. Thus, in this study we identify factors to barriers EPDO crashes while considering the discussed challenges. Those consideration are especially important as in the next step we will employ the modeling results for conducting the cost-benefit analysis. Two main methods were considered in this study to address the discussed challenges including parametric and non-parametric Bayesian hierarchical models. A semiparametric Bayesian approach was used to relax the normality assumption by using a mixture of multivariate Dirichlet prior, defining a flexible nonparametric model for the random effects’ distribution, and using grouping to account for the heterogeneity due to the structure of the dataset. On the other hand, Bayesian hierarchical models with two distributions of Poisson and negative binomial with similar levels of hierarchy were considered. These models were chosen as closest models to the Bayesian semiparametric model. The incorporated models were compared in terms of deviance information criterion (DIC). Results and Discussion: The results highlighted that although the semi-parametric method outperforms the Bayesian hierarchical model with Poisson distribution, the Bayesian hierarchical model with negative binomial (NB) distribution outperform the semi-parametric model. The findings might be related to the severe sparse nature of the EPDO, which cannot optimally be accounted by semiparametric approach, and the model needs more flexibility. Conclusion: It was found that being unrestrained, driving in interstate system, driving in clear weather, light conditions, and driving in a higher traffic all increase the likelihood of EPDO crashes. Also, while some predictors were significant in less accommodative models of semi-parametric or Poisson models, they were not for Negative binomial model.

DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon

AbstractFinding word boundaries in continuous speech is challenging as there is little or no equivalent of a ‘space’ delimiter between words. Popular Bayesian non-parametric models for text segmentation (Goldwater et al., 2006, 2009) use a Dirichlet process to jointly segment sentences and build a lexicon of word types. We introduce DP-Parse, which uses similar principles but only relies on an instance lexicon of word tokens, avoiding the clustering errors that arise with a lexicon of word types. On the Zero Resource Speech Benchmark 2017, our model sets a new speech segmentation state-of-the-art in 5 languages. The algorithm monotonically improves with better input representations, achieving yet higher scores when fed with weakly supervised inputs. Despite lacking a type lexicon, DP-Parse can be pipelined to a language model and learn semantic and syntactic representations as assessed by a new spoken word embedding benchmark. 1

Utility-based Bayesian personalized treatment selection for advanced breast cancer.

A Bayesian method is proposed for personalized treatment selection in settings where data are available from a randomized clinical trial with two or more outcomes. The motivating application is a randomized trial that compared letrozole plus bevacizimab to letrozole alone as first-line therapy for hormone receptor positive advanced breast cancer. The combination treatment arm had larger median progression-free survival time, but also a higher rate of severe toxicities. This suggests that the risk-benefit trade-off between these two outcomes should play a central role in selecting each patient's treatment, particularly since older patients are less likely to tolerate severe toxicities. To quantify the desirability of each possible outcome combination for an individual patient, we elicited from breast cancer oncologists a utility function that varied with age. The utility was used as an explicit criterion for quantifying risk-benefit trade-offs when making personalized treatment selections. A Bayesian nonparametric multivariate regression model with a dependent Dirichlet process prior was fit to the trial data. Under the fitted model, a new patient's treatment can be selected based on the posterior predictive utility distribution. For the breast cancer trial dataset, the optimal treatment depends on the patient's age, with the combination preferable for patients 70 years or younger and the single agent preferable for patients older than 70.

Infinite hidden Markov model for short-term solar irradiance forecasting

Hidden state models are among the most widely used and efficient schemes for solar irradiance modeling in general and forecasting in particular. However, the complexity of such models – in terms of the number of states – is usually needed to be specified a priori. For solar irradiance data this assumption is very difficult to justify.In this paper, an infinite hidden Markov model (InfHMM) is introduced for short-term probabilistic forecasting of solar irradiance, where the assumption of fixed number of states a priori is relaxed and model complexity is determined during the model training. InfHMM is a non-parametric Bayesian model (NPB) indexed with an infinite dimensional parameter space which allows the automatic adaptation of the model to the “correct” complexity. This facilitates the automatic adaptation of the model to all weather conditions and locations. Posterior inference for InfHMM is performed using the Markov chain Monte Carlo algorithm, namely the beam sampler.Data from 13 different sources are used to validate the proposed model and subsequently it is compared to two well-established models in the literature: Markov-chain mixture distribution (MCM) and complete-history persistence ensemble (CH-PeEn) models. Important results are found, that cannot be derived from the existing finite models, such as the variation of the number of states within and across sites. The comparison of the models shows that the InfHMM is more consistent in term of the forecasting horizon.For reproducibility of the methodology presented in this paper, we have provided an R script for the InfHMM as supplementary material.

On a Dirichlet Process Mixture Representation of Phase-Type Distributions

An explicit representation of phase-type distributions as an infinite mixture of Erlang distributions is introduced. The representation unveils a novel and useful connection between a class of Bayesian nonparametric mixture models and phase-type distributions. In particular, this sheds some light on two hot topics, estimation techniques for phase-type distributions, and the availability of closed-form expressions for some functionals related to Dirichlet process mixture models. The power of this connection is illustrated via a posterior inference algorithm to estimate phase-type distributions, avoiding some difficulties with the simulation of latent Markov jump processes, commonly encountered in phase-type Bayesian inference. On the other hand, closed-form expressions for functionals of Dirichlet process mixture models are illustrated with density and renewal function estimation, related to the optimal salmon weight distribution of an aquaculture study.

Non-Parametric Bayesian Covariate-Dependent Multivariate Functional Clustering: An Application to Time-Series Data for Multiple Air Pollutants

Abstract Air pollution is a major threat to public health. Understanding the spatial distribution of air pollution concentration is of great interest to government or local authorities, as it informs about target areas for implementing policies for air quality management. Cluster analysis has been popularly used to identify groups of locations with similar profiles of average levels of multiple air pollutants, efficiently summarising the spatial pattern. This study aimed to cluster locations based on the seasonal patterns of multiple air pollutants incorporating the location-specific characteristics such as socio-economic indicators. For this purpose, we proposed a novel non-parametric Bayesian sparse latent factor model for covariate-dependent multivariate functional clustering. Furthermore, we extend this model to conduct clustering with temporal dependency. The proposed methods are illustrated through a simulation study and applied to time-series data for daily mean concentrations of ozone (O3), nitrogen dioxide (NO2), and fine particulate matter (PM2.5) collected for 25 cities in Canada in 1986–2015.

Nonparametric Bayesian inference for meta-stable conformational dynamics

Analyses of structural dynamics of biomolecules hold great promise to deepen the understanding of and ability to construct complex molecular systems. To this end, both experimental and computational means are available, such as fluorescence quenching experiments or molecular dynamics simulations, respectively. We argue that while seemingly disparate, both fields of study have to deal with the same type of data about the same underlying phenomenon of conformational switching. Two central challenges typically arise in both contexts: (i) the amount of obtained data is large, and (ii) it is often unknown how many distinct molecular states underlie these data. In this study, we build on the established idea of Markov state modeling and propose a generative, Bayesian nonparametric hidden Markov state model that addresses these challenges. Utilizing hierarchical Dirichlet processes, we treat different meta-stable molecule conformations as distinct Markov states, the number of which we then do not have to set a priori. In contrast to existing approaches to both experimental as well as simulation data that are based on the same idea, we leverage a mean-field variational inference approach, enabling scalable inference on large amounts of data. Furthermore, we specify the model also for the important case of angular data, which however proves to be computationally intractable. Addressing this issue, we propose a computationally tractable approximation to the angular model. We demonstrate the method on synthetic ground truth data and apply it to known benchmark problems as well as electrophysiological experimental data from a conformation-switching ion channel to highlight its practical utility.

A Joint Analysis for Field Goal Attempts and Percentages of Professional Basketball Players: Bayesian Nonparametric Resource

Understanding shooting patterns among different players is a fundamental problem in basketball game analyses. In this paper, we quantify the shooting pattern via the field goal attempts and percentages over twelve non-overlapping regions around the front court. A joint Bayesian nonparametric mixture model is developed to find latent clusters of players based on their shooting patterns. We apply our proposed model to learn the heterogeneity among selected players from the National Basketball Association (NBA) games over the 2018–2019 regular season and 2019–2020 bubble season. Thirteen clusters are identified for 2018–2019 regular season and seven clusters are identified for 2019–2020 bubble season. We further examine the shooting patterns of players in these clusters and discuss their relation to players’ other available information. The results shed new insights on the effect of NBA COVID bubble and may provide useful guidance for player’s shot selection and team’s in-game and recruiting strategy planning.

Hierarchical clustering optimizes the tradeoff between compositionality and expressivity of task structures for flexible reinforcement learning

A hallmark of human intelligence, but challenging for reinforcement learning (RL) agents, is the ability to compositionally generalise, that is, to recompose familiar knowledge components in novel ways to solve new problems. For instance, when navigating in a city, one needs to know the location of the destination and how to operate a vehicle to get there, whether it be pedalling a bike or operating a car. In RL, these correspond to the reward function and transition function, respectively. To compositionally generalize, these two components need to be transferable independently of each other: multiple modes of transport can reach the same goal, and any given mode can be used to reach multiple destinations. Yet there are also instances where it can be helpful to learn and transfer entire structures, jointly representing goals and transitions, particularly whenever these recur in natural tasks (e.g., given a suggestion to get ice cream, one might prefer to bike, even in new towns). Prior theoretical work has explored how, in model-based RL, agents can learn and generalize task components (transition and reward functions). But a satisfactory account for how a single agent can simultaneously satisfy the two competing demands is still lacking. Here, we propose a hierarchical RL agent that learns and transfers individual task components as well as entire structures (particular compositions of components) by inferring both through a non-parametric Bayesian model of the task. It maintains a factorised representation of task components through a hierarchical Dirichlet process, but it also represents different possible covariances between these components through a standard Dirichlet process. We validate our approach on a variety of navigation tasks covering a wide range of statistical correlations between task components and show that it can also improve generalisation and transfer in more complex, hierarchical tasks with goal/subgoal structures. Finally, we end with a discussion of our work including how this clustering algorithm could conceivably be implemented by cortico-striatal gating circuits in the brain.