Nonparametric Bayesian Inference Research Articles

An R Package for Nonparametric Inference on Dynamic Populations with Infinitely Many Types.

Fleming-Viot diffusions are widely used stochastic models for population dynamics that extend the celebrated Wright-Fisher diffusions. They describe the temporal evolution of the relative frequencies of the allelic types in an ideally infinite panmictic population, whose individuals undergo random genetic drift and at birth can mutate to a new allelic type drawn from a possibly infinite potential pool, independently of their parent. Recently, Bayesian nonparametric inference has been considered for this model when a finite sample of individuals is drawn from the population at several discrete time points. Previous works have fully described the relevant estimators for this problem, but current software is available only for the Wright-Fisher finite-dimensional case. Here, we provide software for the general case, overcoming some nontrivial computational challenges posed by this setting. The R package FVDDPpkg efficiently approximates the filtering and smoothing distribution for Fleming-Viot diffusions, given finite samples of individuals collected at different times. A suitable Monte Carlo approximation is also introduced in order to reduce the computational cost.

Nonparametric Bayesian estimation in a multidimensional diffusion model with high frequency data

AbstractWe consider nonparametric Bayesian inference in a multidimensional diffusion model with reflecting boundary conditions based on discrete high-frequency observations. We prove a general posterior contraction rate theorem in $$L^2$$ L 2 -loss, which is applied to Gaussian priors. The resulting posteriors, as well as their posterior means, are shown to converge to the ground truth at the minimax optimal rate over Hölder smoothness classes in any dimension. Of independent interest and as part of our proofs, we show that certain frequentist penalized least squares estimators are also minimax optimal.

Inference through innovation processes tested in the authorship attribution task

Urn models for innovation capture fundamental empirical laws shared by several real-world processes. The so-called urn model with triggering includes, as particular cases, the urn representation of the two-parameter Poisson-Dirichlet process and the Dirichlet process, seminal in Bayesian non-parametric inference. In this work, we leverage this connection to introduce a general approach for quantifying closeness between symbolic sequences and test it within the framework of the authorship attribution problem. The method demonstrates high accuracy when compared to other related methods in different scenarios, featuring a substantial gain in computational efficiency and theoretical transparency. Beyond the practical convenience, this work demonstrates how the recently established connection between urn models and non-parametric Bayesian inference can pave the way for designing more efficient inference methods. In particular, the hybrid approach that we propose allows us to relax the exchangeability hypothesis, which can be particularly relevant for systems exhibiting complex correlation patterns and non-stationary dynamics.

Nonparametric Bayesian optimal designs for Unit Exponential regression model with respect to prior processes(with the truncated normal as the base measure)

Nonlinear regression models are extensively applied across various scientific disciplines. It is vital to accurately fit the optimal nonlinear model while considering the biases of the Bayesian optimal design. We present a Bayesian optimal design by utilising the Dirichlet process as a prior. The Dirichlet process serves as a fundamental tool in the exploration of Nonparametric Bayesian inference, offering multiple representations that are well-suited for application. This research paper introduces a novel one-parameter model, referred to as the ’Unit-Exponential distribution’, specifically designed for the unit interval. Additionally, we employ a stick-breaking representation to approximate the D-optimality criterion considering the Dirichlet process as a functional tool. Through this approach, we aim to identify a Nonparametric Bayesian optimal design.

Reconciling shared versus context-specific information in a neural network model of latent causes

It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the “splitting” of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could (1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, (2) capture human data on curriculum effects in schema learning, and (3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.

High accuracy ranging for space debris with spaceborne single photon Lidar.

The increasing risk posed by space debris highlights the need for accurate localization techniques. Spaceborne single photon Lidar (SSPL) offers a promising solution, overcoming the limitations of traditional ground-based systems by providing expansive coverage and superior maneuverability without being hindered by weather, time, or geographic constraints. This study introduces a novel approach leveraging non-parametric Bayesian inference and the Dirichlet process mixture model (DPMM) to accurately determine the distance of space debris in low Earth orbit (LEO), where debris exhibits nonlinear, high dynamic motion characteristics. By integrating extended Kalman filtering (EKF) for range gating, our method captures the temporal distribution of reflected photons, employing Markov chain Monte Carlo (MCMC) for iterative solutions. Experimental outcomes demonstrate our method's superior accuracy over conventional statistical techniques, establishing a clear correlation between radial absolute velocity and ranging error, thus significantly enhancing monostatic space debris localization.

Bayesian Nonparametric Bivariate Survival Regression for Current Status Data

We consider Bayesian nonparametric inference for event time distributions based on current status data. We show that under dependent censoring conventional mixture priors, including the popular Dirichlet process mixture prior, lead to biologically uninterpretable results as they unnaturally skew the probability mass for the event times toward the extremes of the observed data. Simple assumptions on dependent censoring can fix the problem. We then extend the discussion to bivariate current status data with partial ordering of the two outcomes. In addition to dependent censoring, we also exploit some minimal known structure relating the two event times. We design a Markov chain Monte Carlo algorithm for posterior simulation. Applied to a recurrent infection study, the method provides novel insights into how symptoms-related hospital visits are affected by covariates.

Data-Driven Carbon Footprint Management of Electric Vehicles and Emission Abatement in Electricity Networks

Electric vehicles (EVs) are believed to play an important role in mitigating carbon emissions in the transportation sector. However, EVs may still cause carbon emissions in the power sector if they are charged by electricity generated from burning fossil fuels like coal. Researchers have focused on managing emissions on the power generation side. However, the underlying driver of carbon emissions is the demand of consumers. To this end, a probabilistic carbon footprint management strategy is proposed for EVs in this paper. First, the conventional deterministic carbon emission flow model is extended to a probabilistic one (PCEF) to track the carbon footprint of EVs considering various uncertainties based on non-intrusive load monitoring (NILM) and the two-point estimation method (2PEM). Second, a stochastic chance-constrained carbon footprint management model for EV charging is proposed to address the carbon obligation allocation of EVs from the perspective of consumption and provide a technical basis for demand-driven stimulation to reduce carbon emissions. Third, a solution methodology is proposed to solve the formulated chance-constrained problem based on nonparametric Bayesian modeling and inference. The proposed model and methodologies are verified on the IEEE 39-bus system. The feasibility of the proposed PCEF model is validated. According to the simulation result, the computation speed of the proposed PCEF model is improved from 3456.8 seconds to 1341.3 seconds compared with the Monte Carlo simulation by sacrificing the accuracy within 2%. Besides, the proposed emission control strategy can realize a better emission control performance compared with the other state-of-art works. Compared with the base case, the total emission is reduced by 82 tons annually, and the emission reduction rate is increased by nearly twice.

Scalable Bayesian Transport Maps for High-Dimensional Non-Gaussian Spatial Fields

A multivariate distribution can be described by a triangular transport map from the target distribution to a simple reference distribution. We propose Bayesian nonparametric inference on the transport map by modeling its components using Gaussian processes. This enables regularization and uncertainty quantification of the map estimation, while resulting in a closed-form and invertible posterior map. We then focus on inferring the distribution of a nonstationary spatial field from a small number of replicates. We develop specific transport-map priors that are highly flexible and are motivated by the behavior of a large class of stochastic processes. Our approach is scalable to high-dimensional distributions due to data-dependent sparsity and parallel computations. We also discuss extensions, including Dirichlet process mixtures for flexible marginals. We present numerical results to demonstrate the accuracy, scalability, and usefulness of our methods, including statistical emulation of non-Gaussian climate-model output. Supplementary materials for this article are available online.

Plausible presence of new state in neutron stars with masses above [formula omitted

We investigate the neutron star (NS) equation of state (EOS) by incorporating multi-messenger data of GW170817, PSR J0030 + 0451, PSR J0740 + 6620, and state-of-the-art theoretical progresses, including the information from chiral effective field theory (χEFT) and perturbative quantum chromodynamics (pQCD) calculation. Taking advantage of the various structures sampling by a single-layer feed-forward neural network model embedded in the Bayesian nonparametric inference, the structure of NS matter’s sound speed cs is explored in a model-agnostic way. It is found that a peak structure is common in the cs2 posterior, locating at (2.4-4.8)ρsat (nuclear saturation density) and cs2 exceeds c2/3 at 90% credibility. The non-monotonic behavior suggests evidence of the state deviating from the hadronic matter inside the very massive NSs. Assuming the new/exotic state is featured as it is softer than typical hadronic models or even with hyperons, we find that a sizable (⩾10-3M⊙) exotic core, likely made of quark matter, is plausible for the NS with a gravitational mass above about 0.98MTOV, where MTOV represents the maximum gravitational mass of a non-rotating cold NS. The inferred MTOV=(2.18-0.13+0.27)M⊙ (90% credibility) is well consistent with the value of (2.17-0.12+0.15)M⊙ estimated independently with GW170817/GRB 170817A/AT2017gfo assuming a temporary supramassive NS remnant formed after the merger. PSR J0740 + 6620, the most massive NS detected so far, may host a sizable exotic core with a probability of ≈0.36.

Deep bootstrap for Bayesian inference.

For a Bayesian, the task to define the likelihood can be as perplexing as the task to define the prior. We focus on situations when the parameter of interest has been emancipated from the likelihood and is linked to data directly through a loss function. We survey existing work on both Bayesian parametric inference with Gibbs posteriors and Bayesian non-parametric inference. We then highlight recent bootstrap computational approaches to approximating loss-driven posteriors. In particular, we focus on implicit bootstrap distributions defined through an underlying push-forward mapping. We investigate independent, identically distributed (iid) samplers from approximate posteriors that pass random bootstrap weights through a trained generative network. After training the deep-learning mapping, the simulation cost of such iid samplers is negligible. We compare the performance of these deep bootstrap samplers with exact bootstrap as well as MCMC on several examples (including support vector machines or quantile regression). We also provide theoretical insights into bootstrap posteriors by drawing upon connections to model mis-specification. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

Bayesian Nonparametric Panel Markov-Switching GARCH Models

This article proposes Bayesian nonparametric inference for panel Markov-switching GARCH models. The model incorporates series-specific hidden Markov chain processes that drive the GARCH parameters. To cope with the high-dimensionality of the parameter space, the article assumes soft parameter pooling through a hierarchical prior distribution and introduces cross sectional clustering through a Bayesian nonparametric prior distribution. An MCMC posterior approximation algorithm is developed and its efficiency is studied in simulations under alternative settings. An empirical application to financial returns data in the United States is offered with a portfolio performance exercise based on forecasts. A comparison shows that the Bayesian nonparametric panel Markov-switching GARCH model provides good forecasting performances and economic gains in optimal asset allocation.

Resolving Molecular Heterogeneity with Single-Molecule Centrifugation.

For many classes of biomolecules, population-level heterogeneity is an essential aspect of biological function─from antibodies produced by the immune system to post-translationally modified proteins that regulate cellular processes. However, heterogeneity is difficult to fully characterize for multiple reasons: (i) single-molecule approaches are needed to avoid information lost by ensemble-level averaging, (ii) sufficient statistics must be gathered on both a per-molecule and per-population level, and (iii) a suitable analysis framework is required to make sense of a potentially limited number of intrinsically noisy measurements. Here, we introduce an approach that overcomes these difficulties by combining three techniques: a DNA nanoswitch construct to repeatedly interrogate the same molecule, a benchtop centrifuge force microscope (CFM) to obtain thousands of statistics in a highly parallel manner, and a Bayesian nonparametric (BNP) inference method to resolve separate subpopulations with distinct kinetics. We apply this approach to characterize commercially available antibodies and find that polyclonal antibody from rabbit serum is well-modeled by a mixture of three subpopulations. Our results show how combining a spatially and temporally multiplexed nanoswitch-CFM assay with BNP analysis can help resolve complex biomolecular interactions in heterogeneous samples.

Truncated Poisson–Dirichlet approximation for Dirichlet process hierarchical models

The Dirichlet process was introduced by Ferguson in 1973 to use with Bayesian nonparametric inference problems. A lot of work has been done based on the Dirichlet process, making it the most fundamental prior in Bayesian nonparametric statistics. Since the construction of Dirichlet process involves an infinite number of random variables, simulation-based methods are hard to implement, and various finite approximations for the Dirichlet process have been proposed to solve this problem. In this paper, we construct a new random probability measure called the truncated Poisson–Dirichlet process. It sorts the components of a Dirichlet process in descending order according to their random weights, then makes a truncation to obtain a finite approximation for the distribution of the Dirichlet process. Since the approximation is based on a decreasing sequence of random weights, it has a lower truncation error comparing to the existing methods using stick-breaking process. Then we develop a blocked Gibbs sampler based on Hamiltonian Monte Carlo method to explore the posterior of the truncated Poisson–Dirichlet process. This method is illustrated by the normal mean mixture model and Caron–Fox network model. Numerical implementations are provided to demonstrate the effectiveness and performance of our algorithm.

Mixture models with decreasing weights

Decreasing weight prior distributions for mixture models play an important role in nonparametric Bayesian inference. Various random probability measures with decreasing weights have been previously explored and it has been shown that they provide an efficient alternative to the more traditional Dirichlet process mixture model. This ordering of the weights implicitly alleviates the so-called label switching problem, as larger weights correspond to larger groups. A general procedure to define any decreasing weights model based on a characterization of a discrete random variable which also allows for an easy and generic sampling algorithm for estimating the model is provided. An exact representation for the number of expected components is given. Finally, the performance of the mixture model on simulated data sets is investigated numerically.

Scaled Process Priors for Bayesian Nonparametric Estimation of the Unseen Genetic Variation

There is a growing interest in the estimation of the number of unseen features, mostly driven by biological applications. A recent work brought out a peculiar property of the popular completely random measures (CRMs) as prior models in Bayesian nonparametric (BNP) inference for the unseen-features problem: for fixed prior’s parameters, they all lead to a Poisson posterior distribution for the number of unseen features, which depends on the sampling information only through the sample size. CRMs are thus not a flexible prior model for the unseen-features problem and, while the Poisson posterior distribution may be appealing for analytical tractability and ease of interpretability, its independence from the sampling information makes the BNP approach a questionable oversimplification, with posterior inferences being completely determined by the estimation of unknown prior’s parameters. In this article, we introduce the stable-Beta scaled process (SB-SP) prior, and we show that it allows to enrich the posterior distribution of the number of unseen features arising under CRM priors, while maintaining its analytical tractability and interpretability. That is, the SB-SP prior leads to a negative Binomial posterior distribution, which depends on the sampling information through the sample size and the number of distinct features, with corresponding estimates being simple, linear in the sampling information and computationally efficient. We apply our BNP approach to synthetic data and to real cancer genomic data, showing that: (i) it outperforms the most popular parametric and nonparametric competitors in terms of estimation accuracy; (ii) it provides improved coverage for the estimation with respect to a BNP approach under CRM priors. Supplementary materials for this article are available online.

A general Bayesian bootstrap for censored data based on the beta-Stacy process

We introduce a novel procedure to perform Bayesian non-parametric inference with right-censored data, the beta-Stacy bootstrap. This approximates the posterior law of summaries of the survival distribution (e.g. the mean survival time). More precisely, our procedure approximates the joint posterior law of functionals of the beta-Stacy process, a non-parametric process prior that generalizes the Dirichlet process and that is widely used in survival analysis. The beta-Stacy bootstrap generalizes and unifies other common Bayesian bootstraps for complete or censored data based on non-parametric priors. It is defined by an exact sampling algorithm that does not require tuning of Markov Chain Monte Carlo steps. We illustrate the beta-Stacy bootstrap by analyzing survival data from a real clinical trial.

Mediation analysis using Bayesian tree ensembles.

We present a general framework for causal mediation analysis using nonparametric Bayesian methods in the potential outcomes framework. Our model, which we refer to as the Bayesian causal mediation forests model, combines recent advances in Bayesian machine learning using decision tree ensembles, Bayesian nonparametric causal inference, and a Bayesian implementation of the g-formula for computing causal effects. Because of its strong performance on simulated data and because it greatly reduces researcher degrees of freedom, we argue that Bayesian causal mediation forests are highly attractive as a default approach. Of independent interest, we also introduce a new sensitivity analysis technique for mediation analysis with continuous outcomes that is widely applicable. We demonstrate our approach on both simulated and real data sets, and show that our approach obtains low mean squared error and close to nominal coverage of 95% interval estimates, even in highly nonlinear problems on which other methods fail. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Bayesian nonparametric inference for the overlap coefficient: With an application to disease diagnosis.

Diagnostic tests play an important role in medical research and clinical practice. The ultimate goal of a diagnostic test is to distinguish between diseased and nondiseased individuals and before a test is routinely used in practice, it is a pivotal requirement that its ability to discriminate between these two states is thoroughly assessed. The overlap coefficient, which is defined as the proportion of overlap area between two probability density functions, has gained popularity as a summary measure of diagnostic accuracy. We propose two Bayesian nonparametric estimators, based on Dirichlet process mixtures, for estimating the overlap coefficient. We further introduce the covariate‐specific overlap coefficient and develop a Bayesian nonparametric approach based on Dirichlet process mixtures of additive normal models for estimating it. A simulation study is conducted to assess the empirical performance of our proposed estimators. Two illustrations are provided: one concerned with the search for biomarkers of ovarian cancer and another one aimed to assess the age‐specific accuracy of glucose as a biomarker of diabetes.

A Non-parametric Bayesian Learning Model Using Accelerated Variational Inference on Multivariate Beta Mixture Models for Medical Applications

Clustering as an exploratory technique has been a promising approach for performing data analysis. In this paper, we propose a non-parametric Bayesian inference to address clustering problem. This approach is based on infinite multivariate Beta mixture models constructed through the framework of Dirichlet process. We apply an accelerated variational method to learn the model. The motivation behind proposing this technique is that Dirichlet process mixture models are capable to fit the data where the number of components is unknown. For large-scale data, this approach is computationally expensive. We overcome this problem with the help of accelerated Dirichlet process mixture models. Moreover, the truncation is managed using kd-trees. The performance of the model is validated on real medical applications and compared to three other similar alternatives. The results show the outperformance of our proposed framework.