Nonparametric Bayesian Estimation Research Articles

Hierarchical Bayesian pharmacometrics analysis of Baclofen for alcohol use disorder

Alcohol use disorder (AUD), also called alcohol dependence, is a major public health problem, affecting almost of the world’s population. Baclofen, as a selective receptor agonist, has emerged as a promising drug for the treatment of AUD. However, the inter-trial, inter-individual and residual variability in drug concentration over time in a population of patients with AUD is unknown. In this study, we use a hierarchical Bayesian workflow to estimate the parameters of a pharmacokinetic (PK) population model from Baclofen administration to patients with AUD. By monitoring various convergence diagnostics, the probabilistic methodology is first validated on synthetic longitudinal datasets and then applied to infer the PK model parameters based on the clinical data that were retrospectively collected from outpatients treated with oral Baclofen. We show that state-of-the-art advances in automatic Bayesian inference using self-tuning Hamiltonian Monte Carlo (HMC) algorithms provide accurate and decisive predictions on Baclofen plasma concentration at both individual and group levels. Importantly, leveraging the information in prior provides faster computation, better convergence diagnostics, and substantially higher out-of-sample prediction accuracy. Moreover, the root mean squared error as a measure of within-sample predictive accuracy can be misleading for model evaluation, whereas the fully Bayesian information criteria correctly select the true data generating parameters. This study points out the capability of non-parametric Bayesian estimation using adaptive HMC sampling methods for easy and reliable estimation in clinical settings to optimize dosing regimens and efficiently treat AUD.

Nonparametric Bayesian volatility estimation for gamma-driven stochastic differential equations

We study a nonparametric Bayesian approach to estimation of the volatility function of a stochastic differential equation driven by a gamma process. The volatility function is modelled a priori as piecewise constant, and we specify a gamma prior on its values. This leads to a straightforward procedure for posterior inference via an MCMC procedure. We give theoretical performance guarantees (minimax optimal contraction rates for the posterior) for the Bayesian estimate in terms of the regularity of the unknown volatility function. We illustrate the method on synthetic and real data examples.

Estimation of entropy and extropy based on right censored data: A Bayesian non-parametric approach

Abstract Entropy and extropy are central measures in information theory. In this paper, Bayesian non-parametric estimators to entropy and extropy with possibly right censored data are proposed. The approach uses the beta-Stacy process and the difference operator. Examples are presented to illustrate the performance of the estimators.

Bayesian nonparametric inference for the overlap coefficient: With an application to disease diagnosis.

Diagnostic tests play an important role in medical research and clinical practice. The ultimate goal of a diagnostic test is to distinguish between diseased and nondiseased individuals and before a test is routinely used in practice, it is a pivotal requirement that its ability to discriminate between these two states is thoroughly assessed. The overlap coefficient, which is defined as the proportion of overlap area between two probability density functions, has gained popularity as a summary measure of diagnostic accuracy. We propose two Bayesian nonparametric estimators, based on Dirichlet process mixtures, for estimating the overlap coefficient. We further introduce the covariate‐specific overlap coefficient and develop a Bayesian nonparametric approach based on Dirichlet process mixtures of additive normal models for estimating it. A simulation study is conducted to assess the empirical performance of our proposed estimators. Two illustrations are provided: one concerned with the search for biomarkers of ovarian cancer and another one aimed to assess the age‐specific accuracy of glucose as a biomarker of diabetes.

Evaluation of alternative methods for estimating the precision of REML-based estimates of variance components and heritability.

Residual Maximum Likelihood (REML) analysis is the most widely used method to estimate variance components and heritability. This method is based on large sample theory under the assumption that the parameter estimates are asymptotically multivariate normally distributed with covariance matrix given by the inverse of the information matrix. Hence, these sampling variances could be biased if the assumption of asymptotic approximation is incorrect, especially when the sample size is small. Though it is difficult to assess the impact of sample size, an alternative option is to generate a full distribution of the parameters to determine the uncertainty of estimates. In this study, we compared the REML estimates of variance components, heritability and sampling variances of body-weight (BW), body-depth (BD), and condition-factor (K) with those obtained from four sampling-based methods viz., parametric and nonparametric bootstrap, asymptotic sampling and Bayesian estimation. The aim was to understand if a sample size of order 1413 was sufficient to contain adequate information for a reliable asymptotic approximation. The REML solution was close to values obtained from different sampling-based methods indicating that the present sample size was sufficient to estimate reliable genetic variation in different traits with varying heritability. The variance and heritability estimated by a nonparametric bootstrap estimate based on randomization of family effects gave comparable results as evaluated by REML for different traits. Hence, the nonparametric bootstrap estimate can be effectively used to understand whether the sample size is large enough to contain sufficient information under likelihood estimation assumptions.

A test for independence via Bayesian nonparametric estimation of mutual information

Mutual information is a well‐known tool to measure the mutual dependence between variables. In this article, a Bayesian nonparametric estimator of mutual information is established by means of the Dirichlet process and thek‐nearest neighbour distance. As a result, an easy‐to‐implement test of independence is introduced through the relative belief ratio. Several theoretical properties of the approach are presented. The procedure is illustrated through various examples and is compared with its frequentist counterpart.

Shrinkage Priors for Nonparametric Bayesian Prediction of Nonhomogeneous Poisson Processes

We consider nonparametric Bayesian estimation and prediction for nonhomogeneous Poisson process models with unknown intensity functions. We propose a class of improper priors for intensity functions. Nonparametric Bayesian inference with kernel mixture based on the class improper priors is shown to be useful, although improper priors have not been widely used for nonparametric Bayes problems. Several theorems corresponding to those for finite-dimensional independent Poisson models hold for nonhomogeneous Poisson process models with infinite-dimensional parameter spaces. Bayesian estimation and prediction based on the improper priors are shown to be admissible under the Kullback-Leibler loss. Numerical methods for Bayesian inference based on the priors are investigated.

A Bayesian nonparametric estimation to entropy

A Bayesian nonparametric estimator to entropy is proposed. The derivation of the new estimator relies on using the Dirichlet process and adapting the well-known frequentist estimators of Vasicek (Journal of Royal Statistical Society B 38 (1976) 54–59) and Ebrahimi, Pflughoeft and Soofi (Statistics & Probability Letters 20 (1994) 225–234). Several theoretical properties, such as consistency, of the proposed estimator are obtained. The quality of the proposed estimator has been investigated through several examples, in which it exhibits excellent performance.

Nonparametric Bayes subject to overidentified moment conditions

Nonparametric Bayesian estimation subject to overidentified moment equations is a challenge because the support of the posterior is a manifold of lower dimension than the number of model parameters. The manifold therefore has Lebesgue measure zero thus inhibiting the use of the most commonly used Bayesian estimation method: MCMC (Markov Chain Monte Carlo). This study proposes an effective MCMC algorithm and algorithms for estimating scale and the normalizing constant. The algorithms are illustrated with two illustrative applications.

Nonparametric Bayesian background estimation for hyperspectral anomaly detection

We propose an anomaly detection algorithm based on nonparametric Bayesian (NB) background estimation for hyperspectral images. The background is modeled as a Gaussian mixture model (GMM) and the model parameters and order are estimated from the data in an unsupervised manner using nonparametric Bayesian methods called Chinese restaurant process mixtures (CRPM) and truncated Dirichlet process mixtures (TDPM). Since, in anomaly detection, the model parameters are estimated only using the background data in an unsupervised manner, we also propose to use a superpixel-based background data preparation method to obtain a slightly purified background data. In the proposed mixture models, the covariance matrix estimation problem is regularized by defining some conjugate priors to avoid obtaining an ill-conditioned covariance matrix. Finally, the GMM model whose parameters and model order are estimated by the proposed methods is used for anomaly detection. We compare the performances of the proposed methods with those of variational Bayesian, conventional GMM, and collaborative representation-based detector (CRD). Experiments on real hyperspectral data sets show that the proposed methods outperform the detection performance of the other methods.

Nonparametric Bayesian reliability analysis of masked data with dependent competing risks

The characteristic of dependence widely exists among different failure modes of systems, which brings extra difficulty for the reliability analysis. In this paper, a nonparametric Bayesian analysis method is proposed for dependent masked data under accelerated lifetime test with censoring. Using the copula function, the dependence structure is constructed among the competing failure modes which can be viewed as the components in a series system. By establishing the transformational relationship between the subsurvival functions and the survival functions, the estimators of components’ reliability can be derived from the nonparametric Bayesian estimators of the subsurvival functions when a Dirichlet multivariate process prior is considered. A simulation study is given to illustrate the effectiveness of the proposed methods and the influence of the degree of dependence on the performance of the estimation procedure. It is shown that the dependence between failure modes should not be ignored because ignoring the dependence may lead to serious errors. A numerical example based on a life test of rolling ball bearings is presented as an application of the proposed methodology.

A practical introduction to Bayesian estimation of causal effects: Parametric and nonparametric approaches.

Substantial advances in Bayesian methods for causal inference have been made in recent years. We provide an introduction to Bayesian inference for causal effects for practicing statisticians who have some familiarity with Bayesian models and would like an overview of what it can add to causal estimation in practical settings. In the paper, we demonstrate how priors can induce shrinkage and sparsity in parametric models and be used to perform probabilistic sensitivity analyses around causal assumptions. We provide an overview of nonparametric Bayesian estimation and survey their applications in the causal inference literature. Inference in the point-treatment and time-varying treatment settings are considered. For the latter, we explore both static and dynamic treatment regimes. Throughout, we illustrate implementation using off-the-shelf open source software. We hope to leave the reader with implementation-level knowledge of Bayesian causal inference using both parametric and nonparametric models. All synthetic examples and code used in the paper are publicly available on a companion GitHub repository.

Decentralized Multi-agent information-theoretic control for target estimation and localization: finding gas leaks

This article presents a new decentralized multi-agent information-theoretic (DeMAIT) control algorithm for mobile sensors (agents). The algorithm leverages Bayesian estimation and information-theoretic motion planning for efficient and effective estimation and localization of a target, such as a chemical gas leak. The algorithm consists of: (1) a non-parametric Bayesian estimator, (2) an information-theoretic trajectory planner that generates “informative trajectories” for agents to follow, and (3) a controller and collision avoidance algorithm to ensure that each agent follows its trajectory as closely as possible in a safe manner. Advances include the use of a new information-gain metric and its analytical gradient, which do not depend on an infinite series like prior information metrics. Dynamic programming and multi-threading techniques are applied to efficiently compute the mutual information to minimize measurement uncertainty. The estimation and motion planning processes also take into account the dynamics of the sensors and agents. Extensive simulations are conducted to compare the performance between the DeMAIT algorithm to a traditional raster-scanning method and a clustering method with coordination. The main hypothesis that the DeMAIT algorithm outperforms the other two methods is validated, specifically where the average localization success rate for the DeMAIT algorithm is (a) higher and (b) more robust to changes in the source location, robot team size, and search area size than the raster-scanning and clustering methods. Finally, outdoor field experiments are conducted using a team of custom-built aerial robots equipped with gas concentration sensors to demonstrate efficacy of the DeMAIT algorithm to estimate and find the source of a propane gas leak.

Nonparametric Bayesian Modeling and Estimation of Spatial Correlation Functions for Global Data

We provide a nonparametric spectral approach to the modeling of correlation functions on spheres. The sequence of Schoenberg coefficients and their associated covariance functions are treated as random rather than assuming a parametric form. We propose a stick-breaking representation for the spectrum, and show that such a choice spans the support of the class of geodesically isotropic covariance functions under uniform convergence. Further, we examine the first order properties of such representation, from which geometric properties can be inferred, in terms of Hölder continuity, of the associated Gaussian random field. The properties of the posterior, in terms of existence, uniqueness, and Lipschitz continuity, are then inspected. Our findings are validated with MCMC simulations and illustrated using a global data set on surface temperatures.

Estimating stochastic production frontiers: A one-stage multivariate semiparametric Bayesian concave regression method

This paper describes a nonparametric Bayesian estimator for production frontiers that satisfies the axioms of monotonicity and concavity. An inefficiency term that allows for a departure from the homoscedastic prior distributional assumption is jointly estimated in a single stage with cross-sectional data. Our Monte Carlo simulation experiments demonstrate that the frontier and efficiency estimations are computationally competitive, align well with economic theory, and allow for the analysis of larger data sets than existing nonparametric methods. We use the proposed method to investigate Japan’s concrete industry, an important component of the nation’s construction sector, from 2007 to 2010. Our finding of a significant size-weighted inefficiency supports the argument that economic stimuli given to Japan’s concrete industry may result in large losses due to inefficiency.

Nonparametric Bayesian Modeling and Estimation for Renewal Processes

ABSTRACT We propose a flexible approach to modeling for renewal processes. The model is built from a structured mixture of Erlang densities for the renewal process inter-arrival density. The Erlang mixture components have a common scale parameter, and the mixture weights are defined through an underlying distribution function modeled nonparametrically with a Dirichlet process (DP) prior. This model specification enables nonstandard shapes for the inter-arrival time density, including heavy tailed and multimodal densities. Moreover, the choice of the DP centering distribution controls clustering or declustering patterns for the point process, which can therefore be encouraged in the prior specification. Using the analytically available Laplace transforms of the relevant functions, we study the renewal function and the directly related K function, which can be used to infer about clustering or declustering patterns. From a computational point of view, the model structure is attractive as it enables efficient posterior simulation while properly accounting for the likelihood normalizing constant implied by the renewal process. A hierarchical extension of the model allows for the quantification of the impact of different levels of a factor. The modeling approach is illustrated with several synthetic datasets, earthquake occurrences data, and coal-mining disaster data.

Bayesian Nonparametric Estimation ofEx PostVariance

AbstractVariance estimation is central to many questions in finance and economics. Until now ex post variance estimation has been based on infill asymptotic assumptions that exploit high-frequency data. This article offers a new exact finite sample approach to estimating ex post variance using Bayesian nonparametric methods. In contrast to the classical counterpart, the proposed method exploits pooling over high-frequency observations with similar variances. Bayesian nonparametric variance estimators under no noise, heteroskedastic and serially correlated microstructure noise are introduced and discussed. Monte Carlo simulation results show that the proposed approach can increase the accuracy of variance estimation. Applications to equity data and comparison with realized variance and realized kernel estimators are included.

Iterative Bayesian denoising based on variance stabilization using Contourlet Transform with Sharp Frequency Localization: application to EFTEM images

BackgroundDue to the presence of high noise level in tomographic series of energy filtered transmission electron microscopy (EFTEM) images, alignment and 3D reconstruction steps become so difficult. To improve the alignment process which will in turn allow a more accurate and better three dimensional tomography reconstructions, a preprocessing step should be applied to the EFTEM data series.ResultsExperiments with real EFTEM data series at low SNR, show the feasibility and the accuracy of the proposed denoising approach being competitive with the best existing methods for Poisson image denoising. The effectiveness of the proposed denoising approach is thanks to the use of a nonparametric Bayesian estimation in the Contourlet Transform with Sharp Frequency Localization Domain (CTSD) and variance stabilizing transformation (VST). Furthermore, the optimal inverse Anscome transformation to obtain the final estimate of the denoised images, has allowed an accurate tomography reconstruction.ConclusionThe proposed approach provides qualitative information on the 3D distribution of individual chemical elements on the considered sample.

Hierarchical nonparametric survival modeling for demand forecasting with fragmented categorical covariates

AbstractThis paper addresses the problem of data fragmentation when incorporating imbalanced categorical covariates in nonparametric survival models. The problem arises in an application of demand forecasting where certain categorical covariates are important explanatory factors for the diversity of survival patterns but are severely imbalanced in the sense that a large percentage of data segments defined by these covariates have very small sample sizes. Two general approaches, called the class‐based approach and the fusion‐based approach, are proposed to handle the problem. Both reply on judicious utilization of a data segment hierarchy defined by the covariates. The class‐based approach allows certain segments in the hierarchy to have their private survival functions and aggregates the others to share a common survival function. The fusion‐based approach allows all survival functions to borrow and share information from all segments based on their positions in the hierarchy. A nonparametric Bayesian estimator with Dirichlet process priors provides the data‐sharing mechanism in the fusion‐based approach. The hyperparameters in the priors are treated as fixed quantities and learned from data by taking advantage of the data segment hierarchy. The proposed methods are motivated and validated by a case study with real‐world data from an operation of software development service.

Bayesian Inference with M-splines on Spectral Measure of Bivariate Extremes

We consider a Bayesian methodology with M-splines for the spectral measure of a bivariate extreme-value distribution. The tail of a bivariate distribution function F in the max-domain of attraction of an extreme-value distribution function G may be approximated by that of its extreme value attractor. The function G is characterized by a probability measure with expectation equal to 1/2, called the spectral measure, and two extreme-value indices. This spectral measure determines the tail dependence structure of F. The approximation of the spectral measure is proposed thanks to a non-parametric Bayesian estimator that guarantees to fulfill a moment and a shape constraint. The problem of routine calculation of posterior distributions for both coefficients and knots of M-splines is addressed using the Markov chain Monte Carlo (MCMC) simulation technique of reversible jumps.