Nonparametric Bayesian Inference Research Articles

Inferring effective forces for Langevin dynamics using Gaussian processes.

Effective forces derived from experimental or in silico molecular dynamics time traces are critical in developing reduced and computationally efficient descriptions of otherwise complex dynamical problems. This helps motivate why it is important to develop methods to efficiently learn effective forces from time series data. A number of methods already exist to do this when data are plentiful but otherwise fail for sparse datasets or datasets where some regions of phase space are undersampled. In addition, any method developed to learn effective forces from time series data should be minimally a priori committal as to the shape of the effective force profile, exploit every data point without reducing data quality through any form of binning or pre-processing, and provide full credible intervals (error bars) about the prediction for the entirety of the effective force curve. Here, we propose a generalization of the Gaussian process, a key tool in Bayesian nonparametric inference and machine learning, which meets all of the above criteria in learning effective forces for the first time.

Timing matters: real-world effectiveness of early combination of biologic and conventional synthetic disease-modifying antirheumatic drugs for treating newly diagnosed polyarticular course juvenile idiopathic arthritis

ObjectivesTo compare real-world effectiveness of two adaptive treatment strategies of disease-modifying antirheumatic drugs (DMARDs) in treating children with newly diagnosed polyarticular course juvenile idiopathic arthritis (pcJIA): early aggressive use of...

Beyond Whittle: Nonparametric Correction of a Parametric Likelihood with a Focus on Bayesian Time Series Analysis

Nonparametric Bayesian inference has seen a rapid growth over the last decade but only few nonparametric Bayesian approaches to time series analysis have been developed. Most existing approaches use Whittle’s likelihood for Bayesian modelling of the spectral density as the main nonparametric characteristic of stationary time series. It is known that the loss of efficiency using Whittle’s likelihood can be substantial. On the other hand, parametric methods are more powerful than nonparametric methods if the observed time series is close to the considered model class but fail if the model is misspecified. Therefore, we suggest a nonparametric correction of a parametric likelihood that takes advantage of the efficiency of parametric models while mitigating sensitivities through a nonparametric amendment. We use a nonparametric Bernstein polynomial prior on the spectral density with weights induced by a Dirichlet process and prove posterior consistency for Gaussian stationary time series. Bayesian posterior computations are implemented via an MH-within-Gibbs sampler and the performance of the nonparametrically corrected likelihood for Gaussian time series is illustrated in a simulation study and in three astronomy applications, including estimating the spectral density of gravitational wave data from the Advanced Laser Interferometer Gravitational-wave Observatory (LIGO).

Consistency of Bayesian nonparametric inference for discretely observed jump diffusions

We introduce verifiable criteria for weak posterior consistency of Bayesian nonparametric inference for jump diffusions with unit diffusion coefficient and uniformly Lipschitz drift and jump coefficients in arbitrary dimension. The criteria are expressed in terms of coefficients of the SDEs describing the process, and do not depend on intractable quantities such as transition densities. We also show that priors built from discrete nets, wavelet expansions, and Dirichlet mixture models satisfy our conditions. This generalises known results by incorporating jumps into previous work on unit diffusions with uniformly Lipschitz drift coefficients.

Efficient nonparametric Bayesian inference for $X$-ray transforms

We consider the statistical inverse problem of recovering a function $f:M\to \mathbb{R}$, where $M$ is a smooth compact Riemannian manifold with boundary, from measurements of general $X$-ray transforms $I_{a}(f)$ of $f$, corrupted by additive Gaussian noise. For $M$ equal to the unit disk with “flat” geometry and $a=0$ this reduces to the standard Radon transform, but our general setting allows for anisotropic media $M$ and can further model local “attenuation” effects—both highly relevant in practical imaging problems such as SPECT tomography. We study a nonparametric Bayesian inference method based on standard Gaussian process priors for $f$. The posterior reconstruction of $f$ corresponds to a Tikhonov regulariser with a reproducing kernel Hilbert space norm penalty that does not require the calculation of the singular value decomposition of the forward operator $I_{a}$. We prove Bernstein–von Mises theorems for a large family of one-dimensional linear functionals of $f$, and they entail that posterior-based inferences such as credible sets are valid and optimal from a frequentist point of view. In particular we derive the asymptotic distribution of smooth linear functionals of the Tikhonov regulariser, which attains the semiparametric information lower bound. The proofs rely on an invertibility result for the “Fisher information” operator $I_{a}^{*}I_{a}$ between suitable function spaces, a result of independent interest that relies on techniques from microlocal analysis. We illustrate the performance of the proposed method via simulations in various settings.

A Bayesian nonparametric causal inference model for synthesizing randomized clinical trial and real-world evidence.

With the wide availability of various real-world data (RWD), there is an increasing interest in synthesizing information from both randomized clinical trials and RWD for health-care decision makings. The task of addressing study-specific heterogeneities is one of the most difficult challenges in synthesizing data from disparate sources. Bayesian hierarchical models with nonparametric extension provide a powerful and convenient platform that formalizes the information borrowing strength across the sources. In this paper, we propose a propensity score-based Bayesian nonparametric Dirichlet process mixture model that summarizes subject-level information from randomized and registry studies to draw inference on the causal treatment effect. Simulation studies are conducted to evaluate the model performance under different scenarios. In addition, we demonstrate the proposed method using data from a clinical study on angiotensin converting enzyme inhibitor for treating congestive heart failure.

Bernstein–von Mises theorems for statistical inverse problems II: compound Poisson processes

We study nonparametric Bayesian statistical inference for the parameters governing a pure jump process of the form $$Y_t = \sum_{k=1}^{N(t)} Z_k,~~~ t \ge 0,$$ where $N(t)$ is a standard Poisson process of intensity $\lambda$, and $Z_k$ are drawn i.i.d.~from jump measure $\mu$. A high-dimensional wavelet series prior for the L\'evy measure $\nu = \lambda \mu$ is devised and the posterior distribution arises from observing discrete samples $Y_\Delta, Y_{2\Delta}, \dots, Y_{n\Delta}$ at fixed observation distance $\Delta$, giving rise to a nonlinear inverse inference problem. We derive contraction rates in uniform norm for the posterior distribution around the true L\'evy density that are optimal up to logarithmic factors over H\"older classes, as sample size $n$ increases. We prove a functional Bernstein-von Mises theorem for the distribution functions of both $\mu$ and $\nu$, as well as for the intensity $\lambda$, establishing the fact that the posterior distribution is approximated by an infinite-dimensional Gaussian measure whose covariance structure is shown to attain the information lower bound for this inverse problem. As a consequence posterior based inferences, such as nonparametric credible sets, are asymptotically valid and optimal from a frequentist point of view.

A Stochastic Hybrid Structure for Predicting Disturbances in Mixed Automated and Human-Driven Vehicular Scenarios

Abstract In this work, we introduce a stochastic prediction method which can be utilized in applications such as cooperative adaptive cruise control (CACC) to predict interfering vehicles’ movements. One of the main criteria in the design of automated vehicle systems is their robustness against the disturbances resulted from the non-homogeneity of the vehicular environment. The non-homogeneity is mainly due to the human-driven and automated/autonomous vehicles co-existence. It is therefore imperative for the automated applications to be designed with the capability of handling the uncertain behaviors of human-driven vehicles in a robust manner. This paper presents a method for vehicle movements time-series forecasting using a powerful non-parametric Bayesian inference method, namely Gaussian Processes. The proposed methodology is evaluated using realistic vehicle trajectory data from NGSIM dataset and is shown to provide more accurate results compared to baseline methods that use constant velocity coasting.

Bayesian nonparametric inference for the overlap of daily animal activity patterns

The study of the interaction among species is an active area of research in Ecology. In particular, it is of interest to evaluate the overlap of their ecological niches. Temporal activity is one of the niche’s axes most commonly used to explore ecological segregation among animal species, and many contributions focus on the overlap of this variable. Once the information of the temporal activity is obtained in the wild, the data is treated as a random sample. There exist different methods to estimate the overlap. Specifically, in the case of two species, one possibility is to estimate the density of the temporal activity of each species and then evaluate the overlap between these density functions. This leads naturally to the analysis of circular data. Most of the procedures currently in use impose some rather restrictive assumptions on the probabilistic models used to describe the phenomena, and only provide approximate measures of the uncertainty involved in the process. In this article, we propose a Bayesian nonparametric approach which incorporates a well-defined noninformative prior. We take advantage of the data structure to define such a prior in terms of the predictive distribution. To the best of our knowledge, this is a novel approach. Our procedure is compared with a well-known method using simulated data, and applied to the analysis of real camera-trap data concerning two mammalian species from the El Triunfo biosphere reserve (Chiapas, Mexico).

Non-parametric Bayesian inference through MCMC method for Y-linked two-sex branching processes with blind choice

ABSTRACTDirichlet-process-based non-parametric Bayesian inference is developed for a Y-linked two-sex branching process with blind choice. This stochastic model is suitable for analysing the evolution of the number of carriers of two alleles of a Y-linked gene in a two-sex monogamous population where each female chooses her partner from among the male population without caring about his type (i.e. the allele he carries). The only data assumed to be available are the total number of females and males (regardless of their types) up to some generation and the numbers of each type of male in the last generation. A simulation method which is based on a Dirichlet process and a Gibbs sampler is developed to estimate the posterior distributions of the model's main parameters. Finally, the computational efficiency of the algorithm is illustrated with example simulations and an application to real data.

Equivalence of weak and strong modes of measures on topological vector spaces

A strong mode of a probability measure on a normed space X can be defined as a point u such that the mass of the ball centred at u uniformly dominates the mass of all other balls in the small-radius limit. Helin and Burger weakened this definition by considering only pairwise comparisons with balls whose centres differ by vectors in a dense, proper linear subspace E of X, and posed the question of when these two types of modes coincide. We show that, in a more general setting of metrisable vector spaces equipped with measures that are finite on bounded sets, the density of E and a uniformity condition suffice for the equivalence of these two types of modes. We accomplish this by introducing a new, intermediate type of mode. We also show that these modes can be inequivalent if the uniformity condition fails. Our results shed light on the relationships between among various notions of maximum a posteriori estimator in non-parametric Bayesian inference.

Non-parametric Bayesian inference of strategies in repeated games

Inferring underlying cooperative and competitive strategies from human behaviour in repeated games is important for accurately characterizing human behaviour and understanding how people reason strategically. Finite automata, a bounded model of computation, have been extensively used to compactly represent strategies for these games and are a standard tool in game theoretic analyses. However, inference over these strategies in repeated games is challenging since the number of possible strategies grows exponentially with the number of repetitions yet behavioural data are often sparse and noisy. As a result, previous approaches start by specifying a finite hypothesis space of automata that does not allow for flexibility. This limitation hinders the discovery of novel strategies that may be used by humans but are not anticipated a priori by current theory. Here we present a new probabilistic model for strategy inference in repeated games by exploiting non‐parametric Bayesian modelling. With simulated data, we show that the model is effective at inferring the true strategy rapidly and from limited data, which leads to accurate predictions of future behaviour. When applied to experimental data of human behaviour in a repeated prisoner's dilemma, we uncover strategies of varying complexity and diversity.

Bayesian non-parametric inference for $\Lambda$-coalescents: Posterior consistency and a parametric method

We investigate Bayesian non-parametric inference of the $\Lambda$-measure of $\Lambda$-coalescent processes with recurrent mutation, parametrised by probability measures on the unit interval. We give verifiable criteria on the prior for posterior consistency when observations form a time series, and prove that any non-trivial prior is inconsistent when all observations are contemporaneous. We then show that the likelihood given a data set of size $n \in \mathbb{N}$ is constant across $\Lambda$-measures whose leading $n - 2$ moments agree, and focus on inferring truncated sequences of moments. We provide a large class of functionals which can be extremised using finite computation given a credible region of posterior truncated moment sequences, and a pseudo-marginal Metropolis-Hastings algorithm for sampling the posterior. Finally, we compare the efficiency of the exact and noisy pseudo-marginal algorithms with and without delayed acceptance acceleration using a simulation study.

A data-driven photometric stereo algorithm

Representing the complicated reflectance of non-Lambertian material is one of the difficulty in photometric stereo. This paper presents a Gaussian process based data-driven photometric stereo method to address this problem. The method makes use of Gaussian process, a non-parametric Bayesian inference model, to establish a global plausible model to represent non-linear reflectance property of non-Lambertian material based on measured bidirectional reflectance distribution functions. The statistic nature of the Gaussian process makes the method has much better flexibility and accuracy in describing the complicated reflectance of various non-Lambterian materials so that a better estimation accuracy can be achieved for a photometric stereo system. The validity of the proposed method is verified by both computer simulation and real experiment. The results show that the proposed method has much better accuracy and efficiency in estimating surface normal than other existing methods.

Parallel probabilistic graphical model approach for nonparametric Bayesian inference

We propose an efficient uncertainty quantification framework that makes use of multiple probabilistic graphical models to yield a nonparametric Gaussian mixture description of the target probability distribution. The methodology is indeed generic, but this work focuses on its application to the particular class of the inference problems arising from the hidden Markov process and the associated observations in a sequence. The implementation procedure is demonstrated with the dynamical system models in both low and high dimension. In case of the low dimension, it is shown that the usual factor graph for the sequential data can be used to produce a very accurate approximate solution. However, for high dimensional systems, a new family of the factor graphs are developed in order to achieve an effective dimension reduction and to facilitate a synergetic application together with multiple graphs in addressing the Bayesian data assimilation. As a result, a new paradigm for the probabilistic filtering and smoothing emerges, and the applicability of the graphical model approach has been broadened.

Dynamic Clustering Algorithms via Small-Variance Analysis of Markov Chain Mixture Models.

Bayesian nonparametrics are a class of probabilistic models in which the model size is inferred from data. A recently developed methodology in this field is small-variance asymptotic analysis, a mathematical technique for deriving learning algorithms that capture much of the flexibility of Bayesian nonparametric inference algorithms, but are simpler to implement and less computationally expensive. Past work on small-variance analysis of Bayesian nonparametric inference algorithms has exclusively considered batch models trained on a single, static dataset, which are incapable of capturing time evolution in the latent structure of the data. This work presents a small-variance analysis of the maximum a posteriori filtering problem for a temporally varying mixture model with a Markov dependence structure, which captures temporally evolving clusters within a dataset. Two clustering algorithms result from the analysis: D-Means, an iterative clustering algorithm for linearly separable, spherical clusters; and SD-Means, a spectral clustering algorithm derived from a kernelized, relaxed version of the clustering problem. Empirical results from experiments demonstrate the advantages of using D-Means and SD-Means over contemporary clustering algorithms, in terms of both computational cost and clustering accuracy.

Bayesian nonparametric inference for panel count data with an informative observation process.

In this paper, the panel count data analysis for recurrent events is considered. Such analysis is useful for studying tumor or infection recurrences in both clinical trial and observational studies. A bivariate Gaussian Cox process model is proposed to jointly model the observation process and the recurrent event process. Bayesian nonparametric inference is proposed for simultaneously estimating regression parameters, bivariate frailty effects, and baseline intensity functions. Inference is done through Markov chain Monte Carlo, with fully developed computational techniques. Predictive inference is also discussed under the Bayesian setting. The proposed method is shown to be efficient via simulation studies. A clinical trial dataset on skin cancer patients is analyzed to illustrate the proposed approach.

Search for quasi-periodic signals in magnetar giant flares

Quasi-periodic oscillations (QPOs) discovered in the decaying tails of giant flares of magnetars are believed to be torsional oscillations of neutron stars. These QPOs have a high potential to constrain properties of high-density matter. In search for quasi-periodic signals, we study the light curves of the giant flares of SGR 1806-20 and SGR 1900+14, with a non-parametric Bayesian signal inference method called D3PO. The D3PO algorithm models the raw photon counts as a continuous flux and takes the Poissonian shot noise as well as all instrument effects into account. It reconstructs the logarithmic flux and its power spectrum from the data. Using this fully noise-aware method, we do not confirm previously reported frequency lines at ν ≳ 17 Hz because they fall into the noise-dominated regime. However, we find two new potential candidates for oscillations at 9.2 Hz (SGR 1806-20) and 7.7 Hz (SGR 1900+14). If these are real and the fundamental magneto-elastic oscillations of the magnetars, current theoretical models would favour relatively weak magnetic fields B̅ ~ 6× 1013–3 × 1014 G (SGR 1806-20) and a relatively low shear velocity inside the crust compared to previous findings.

Nonparametric Bayesian inference for mean residual life functions in survival analysis.

Modeling and inference for survival analysis problems typically revolves around different functions related to the survival distribution. Here, we focus on the mean residual life (MRL) function, which provides the expected remaining lifetime given that a subject has survived (i.e. is event-free) up to a particular time. This function is of direct interest in reliability, medical, and actuarial fields. In addition to its practical interpretation, the MRL function characterizes the survival distribution. We develop general Bayesian nonparametric inference for MRL functions built from a Dirichlet process mixture model for the associated survival distribution. The resulting model for the MRL function admits a representation as a mixture of the kernel MRL functions with time-dependent mixture weights. This model structure allows for a wide range of shapes for the MRL function. Particular emphasis is placed on the selection of the mixture kernel, taken to be a gamma distribution, to obtain desirable properties for the MRL function arising from the mixture model. The inference method is illustrated with a data set of two experimental groups and a data set involving right censoring. The supplementary material available at Biostatistics online provides further results on empirical performance of the model, using simulated data examples.

A nonparametric Bayesian approach to copula estimation

ABSTRACTWe propose a novel Dirichlet-based Pólya tree (D-P tree) prior on the copula and based on the D-P tree prior, a nonparametric Bayesian inference procedure. Through theoretical analysis and simulations, we are able to show that the flexibility of the D-P tree prior ensures its consistency in copula estimation, thus able to detect more subtle and complex copula structures than earlier nonparametric Bayesian models, such as a Gaussian copula mixture. Furthermore, the continuity of the imposed D-P tree prior leads to a more favourable smoothing effect in copula estimation over classic frequentist methods, especially with small sets of observations. We also apply our method to the copula prediction between the S&P 500 index and the IBM stock prices during the 2007–08 financial crisis, finding that D-P tree-based methods enjoy strong robustness and flexibility over classic methods under such irregular market behaviours.