Bayesian Nonparametric Approach Research Articles

Nonparametric Bayesian approach to extrapolation problems in configuration interaction methods

We propose a non-parametric extrapolation method based on constrained Gaussian processes for configuration interaction methods. Our method has many advantages: (i) applicability to small data sets such as results of {\it ab initio} methods, (ii) flexibility to incorporate constraints, which are guided by physics, into the extrapolation model, (iii) providing predictions with quantified extrapolation uncertainty, etc. In the present study, we show an application to the extrapolation needed in full configuration interaction method as an example.

A non-parametric Bayesian change-point method for recurrent events

This paper proposes a non-parametric Bayesian approach to detect the change-points of intensity rates in the recurrent-event context and cluster subjects by the change-points. Recurrent events are commonly observed in medical and engineering research. The event counts are assumed to follow a non-homogeneous Poisson process with piecewise-constant intensity functions. We propose a Dirichlet process mixture model to accommodate heterogeneity in subject-specific change-points. The proposed approach provides an objective way of clustering subjects based on the change-points without the need of pre-specified number of latent clusters or model selection procedure. A simulation study shows that the proposed model outperforms the existing Bayesian finite mixture model in detecting the number of latent classes. The simulation study also suggests that the proposed method is robust to the violation of model assumptions. We apply the proposed methodology to the Naturalistic Teenage Driving Study data to assess the change in driving risk and detect subgroups of drivers.

Nonparametric Bayesian estimation of a Hölder continuous diffusion coefficient

We consider a nonparametric Bayesian approach to estimate the diffusion coefficient of a stochastic differential equation given discrete time observations over a fixed time interval. As a prior on the diffusion coefficient, we employ a histogram-type prior with piecewise constant realisations on bins forming a partition of the time interval. Specifically, these constants are realizations of independent inverse Gamma distributed randoma variables. We justify our approach by deriving the rate at which the corresponding posterior distribution asymptotically concentrates around the data-generating diffusion coefficient. This posterior contraction rate turns out to be optimal for estimation of a H\"older-continuous diffusion coefficient with smoothness parameter $0<\lambda\leq 1.$ Our approach is straightforward to implement, as the posterior distributions turn out to be inverse Gamma again, and leads to good practical results in a wide range of simulation examples. Finally, we apply our method on exchange rate data sets.

Correlation based snapshot models of the archeomagnetic field

SUMMARY For the time stationary global geomagnetic field, a new modelling concept is presented. A Bayesian non-parametric approach provides realistic location dependent uncertainty estimates. Modelling related variabilities are dealt with systematically by making little subjective a priori assumptions. Rather than parametrizing the model by Gauss coefficients, a functional analytic approach is applied. The geomagnetic potential is assumed a Gaussian process to describe a distribution over functions. A priori correlations are given by an explicit kernel function with non-informative dipole contribution. A refined modelling strategy is proposed that accommodates non-linearities of archeomagnetic observables: First, a rough field estimate is obtained considering only sites that provide full field vector records. Subsequently, this estimate supports the linearization that incorporates the remaining incomplete records. The comparison of results for the archeomagnetic field over the past 1000 yr is in general agreement with previous models while improved model uncertainty estimates are provided.

ParticleMDI: particle Monte Carlo methods for the cluster analysis of multiple datasets with applications to cancer subtype identification

We present a novel nonparametric Bayesian approach for performing cluster analysis in a context where observational units have data arising from multiple sources. Our approach uses a particle Gibbs sampler for inference in which cluster allocations are jointly updated using a conditional particle filter within a Gibbs sampler, improving the mixing of the MCMC chain. We develop several approaches to improving the computational performance of our algorithm. These methods can achieve greater than an order-of-magnitude improvement in performance at no cost to accuracy and can be applied more broadly to Bayesian inference for mixture models with a single dataset. We apply our algorithm to the discovery of risk cohorts amongst 243 patients presenting with kidney renal clear cell carcinoma, using samples from the Cancer Genome Atlas, for which there are gene expression, copy number variation, DNA methylation, protein expression and microRNA data. We identify 4 distinct consensus subtypes and show they are prognostic for survival rate (p < 0.0001).

Nonparametric Bayesian Estimation of a HHlder Continuous Diffusion Coefficient

We consider a nonparametric Bayesian approach to estimate the diffusion coefficient of a stochastic differential equation given discrete time observations over a fixed time interval. As a prior on the diffusion coefficient, we employ a histogram-type prior with piecewise constant realisations on bins forming a partition of the time interval. Specifically, these constants are realizations of independent inverse Gamma distributed randoma variables. We justify our approach by deriving the rate at which the corresponding posterior distribution asymptotically concentrates around the data-generating diffusion coefficient. This posterior contraction rate turns out to be optimal for estimation of a Holder-continuous diffusion coefficient with smoothness parameter 0<λ≤1. Our approach is straightforward to implement, as the posterior distributions turn out to be inverse Gamma again, and leads to good practical results in a wide range of simulation examples. Finally, we apply our method on exchange rate data sets.

WIKS: a general Bayesian nonparametric index for quantifying differences between two populations

A key problem in many research investigations is to decide whether two samples have the same distribution. Numerous statistical methods have been devoted to this issue, but only few considered a Bayesian nonparametric approach. In this paper, we propose a novel nonparametric Bayesian index (WIKS) for quantifying the difference between two populations $$P_1$$ and $$P_2$$ , which is defined by a weighted posterior expectation of the Kolmogorov–Smirnov distance between $$P_1$$ and $$P_2$$ . We present a Bayesian decision-theoretic argument to support the use of WIKS index and a simple algorithm to compute it. Furthermore, we prove that WIKS is a statistically consistent procedure and that it controls the significance level uniformly over the null hypothesis, a feature that simplifies the choice of cutoff values for taking decisions. We present a real data analysis and an extensive simulation study showing that WIKS is more powerful than competing approaches under several settings.

A Bayesian nonparametric approach for evaluating the causal effect of treatment in randomized trials with semi-competing risks.

We develop a Bayesian nonparametric (BNP) approach to evaluate the causal effect of treatment in a randomized trial where a nonterminal event may be censored by a terminal event, but not vice versa (i.e., semi-competing risks). Based on the idea of principal stratification, we define a novel estimand for the causal effect of treatment on the nonterminal event. We introduce identification assumptions, indexed by a sensitivity parameter, and show how to draw inference using our BNP approach. We conduct simulation studies and illustrate our methodology using data from a brain cancer trial. The R code implementing our model and algorithm is available for download at https://github.com/YanxunXu/BaySemiCompeting.

A Bayesian Nonparametric Latent Approach for Score Distributions in Test Equating

Equating is a family of statistical models and methods used to adjust scores on different test forms so that they can be comparable and used interchangeably. Equated scores are obtained estimating the equating transformation function, which maps the scores on the scale of one test form into their equivalents on the scale of other one. All the statistical models that have been proposed for estimating this function are based on continuous approximations of the score distributions, leading to equated scores lying on a continuous scale even though score scales are usually subsets of the integer numbers (e.g., the total number of correct answers). In this article, we develop a new equating method from which equated scores defined on the original discrete scale are obtained. Considering scores as ordinal random variables, we propose a continuous latent variable formulation to perform an equipercentile-like equating based on a Bayesian nonparametric model for score distributions. The proposed model is applied to simulated and real data collected under an equivalent group design. Some methods to assess the performance of our model are also discussed. Compared with discrete versions of equated scored obtained from traditional equating methods, the results show that the proposed method has better performance.

Robust Small Area Estimation: An Overview

A small area typically refers to a subpopulation or domain of interest for which a reliable direct estimate, based only on the domain-specific sample, cannot be produced due to small sample size in the domain. While traditional small area methods and models are widely used nowadays, there have also been much work and interest in robust statistical inference for small area estimation (SAE). We survey this work and provide a comprehensive review here. We begin with a brief review of the traditional SAE methods. We then discuss SAE methods that are developed under weaker assumptions and SAE methods that are robust in certain ways, such as in terms of outliers or model failure. Our discussion also includes topics such as nonparametric SAE methods, Bayesian approaches, model selection and diagnostics, and missing data. A brief review of software packages available for implementing robust SAE methods is also given.

A new predictive model for the outlet turbidity in micro-irrigation sand filters fed with effluents using Gaussian process regression

Sand media filters used in microirrigation systems must remove suspended particle load for avoiding emitter physical clogging. Turbidity is a parameter related to suspended particle load that it is easy and quick to measure and it is also included in some guidelines for reusing effluents in irrigation. Currently, there are not sufficiently accurate models available to predict outlet turbidity for sand filters, which would be useful for both irrigators and engineers. The aim of this study was to obtain a predictive model able to perform an early detection of the sand filter outlet value of turbidity. This study presents a powerful and effective Bayesian nonparametric approach, termed Gaussian process regression (GPR) model, for predicting the output turbidity (Turbo) from data corresponding to 637 samples of different sand filters using reclaimed effluent. This optimization technique involves kernel parameter setting in the GPR training procedure, which significantly influences the regression accuracy. To this end, the most important parameters of this process are monitored and analyzed: type of filter, height of the filter bed (H), filtration velocity (v) and filter inlet values of the electrical conductivity (CEi), dissolved oxygen (DOi), pHi, turbidity (Turbi) and water temperature (Ti). The results of the present study are two-fold. In the first place, the significance of each variable on the filtration is presented through the model. Secondly, a model for forecasting the outlet turbidity was obtained with success. Indeed, regression with optimal hyperparameters was performed and a coefficient of determination equal to 0.8921 for outlet turbidity was obtained when this new predictive GPR–based model was applied to the experimental dataset. The agreement between experimental data and the model confirmed the good performance of the latter.

Extraction of Nonlinear Synergies for Proportional and Simultaneous Estimation of Finger Kinematics.

Proportional and simultaneous est-imation of finger kinematics from surface EMG based on the assumption that there exists a correlation between muscle activations and finger kinematics in low dimensional space. We employ Manifold Relevance Determination (MRD), a multi-view learning model with a nonparametric Bayesian approach, to extract the nonlinear muscle and kinematics synergies and the relationship between them by studying muscle activations (input-space) together with the finger kinematics (output-space). This study finds that there exist muscle synergies which are associated with kinematic synergies. The acquired nonlinear synergies and the association between them has further been utilized for the estimation of finger kinematics from muscle activation inputs, and the proposed approach has outperformed other commonly used linear and nonlinear regression approaches with an average correlation coefficient of 0.91±0.03. There exists an association between muscle and kinematic synergies which can be used for the proportional and simultaneous estimation of finger kinematics from the muscle activation inputs. The findings of this study not only presents a viable approach for accurate and intuitive myoelectric control but also provides a new perspective on the muscle synergies in the motor control community.

A Bayesian Latent Class Model to Predict Kidney Obstruction in the Absence of Gold Standard

Kidney obstruction, if untreated in a timely manner, can lead to irreversible loss of renal function. A widely used technology for evaluations of kidneys with suspected obstruction is diuresis renography. However, it is generally very challenging for radiologists who typically interpret renography data in practice to build high level of competency due to the low volume of renography studies and insufficient training. Another challenge is that there is currently no gold standard for detection of kidney obstruction. Seeking to develop a computer-aided diagnostic (CAD) tool that can assist practicing radiologists to reduce errors in the interpretation of kidney obstruction, a recent study collected data from diuresis renography, interpretations on the renography data from highly experienced nuclear medicine experts as well as clinical data. To achieve the objective, we develop a statistical model that can be used as a CAD tool for assisting radiologists in kidney interpretation. We use a Bayesian latent class modeling approach for predicting kidney obstruction through the integrative analysis of time-series renogram data, expert ratings, and clinical variables. A nonparametric Bayesian latent factor regression approach is adopted for modeling renogram curves in which the coefficients of the basis functions are parameterized via the factor loadings dependent on the latent disease status and the extended latent factors that can also adjust for clinical variables. A hierarchical probit model is used for expert ratings, allowing for training with rating data from multiple experts while predicting with at most one expert, which makes the proposed model operable in practice. An efficient MCMC algorithm is developed to train the model and predict kidney obstruction with associated uncertainty. We demonstrate the superiority of the proposed method over several existing methods through extensive simulations. Analysis of the renal study also lends support to the usefulness of our model as a CAD tool to assist less experienced radiologists in the field. Supplementary materials for this article are available online.

Platoon Separation Strategy Optimization Method Based on Deep Cognition of a Driver’s Behavior at Signalized Intersections

Semantic understanding of drivers’ behavior features at intersections plays a pivotal role in the proper decision-making of a platoon. This paper presents a flexible framework to automatically extract the driver’s driving features from observed temporal sequences of driving raw data and traffic light information. An approach, which contains two key sub-problems, is proposed to select the separated vehicles from the platoon in the vicinity of the intersection. Then, the first sub-problem, accurately capturing the drivers’ driving behavior features under the impact of traffic lights, is addressed by using the Bayesian nonparametric approach, which could segment drivers’ driving raw data temporal sequences into small analytically interpretable components (called driving primitives) without using prior knowledge. In addition, the extracted driving primitives are used to obtain the vehicle separation strategy (which is also the second sub-problem) by considering safety, efficiency, and energy consumption. Finally, 200 groups of raw data of human-driven vehicles approaching the intersection are used to validate the effectiveness of the proposed primitive-based framework. Experimental results demonstrate that the acceleration indeterminacy of separated vehicles could be decreased 37%-72% by segmenting the captured driving behavior features into $3\times 15$ patterns. Moreover, the vehicle separation strategy could not only increase the efficiency, but also the safety, and the energy consumption could be decreased.

Advanced Driver Assistance Strategies for a Single-Vehicle Overtaking a Platoon on the Two-Lane Two-Way Road

Recently, the ever-increasing vehicle population has become a severe challenge to traffic safety, especially the problem of a single-vehicle overtaking a platoon on the Two-Lane Two-Way (TLTW) road. Platooning has the potential to improve traffic efficiency and safety. However, there exists a perilous situation of “Neither overtake nor give up” when the single-vehicle overtakes a platoon on the TLTW road. This paper presents a flexible framework to automatically filter a large quantity of Advanced Driver Assistance Strategies (ADAS) and select the most suitable driver assistance information for the single-vehicle overtakes a platoon on the TLTW road. A step-by-step Single Vehicle Overtakes Platoon (SVOP) algorithm is designed to generate the coarse ADAS, which had given plenty of consideration to the vehicle safety, traffic efficiency, and driving comfort. Then, this paper obtains the raw data about the single-vehicle overtakes a platoon on the TLTW by using CARLA, which can help us to get 20 drivers’ upper and down boundaries of both velocity and acceleration. In addition, the extracted ranges of velocity and acceleration are used to quantitatively analyse the drivers’ driving features and filter the ADAS information. Finally, a Bayesian nonparametric approach is developed to segment driver’s driving raw data temporal sequences into small analytically interpretable components without using prior knowledge. So that the accurately overtaking characteristics can be obtained, and the ADAS can be further filtered. Experimental results demonstrate that the obtained coarse ADAS are only valid in theory but not acceptable by most of the drivers. Nonetheless, by leveraging the nonparametric Bayes algorithm, the driver’s overtaking behavior can be divided into different primitives, from which some could obtain the driver’s acceptance range for the velocity and acceleration. 92.3% ~ 94.78% invalid SVOP ADAS could be filtered out by leveraging the primitive-based SVOP approach. Thus, after filtering, the overtaking scheme is the most acceptable strategy for drivers.

Non-Intrusive Appliance Identification with Appliance-Specific Networks

The problem of noninstrusive load monitoring (NILM) is usually formulated as a single-channel blind source separation task, whose successful solution enable fast and convenient load identification and energy disaggregation. When applied at test time, NILM algorithms aim to identify the operating characteristics of individual appliances from an aggregate power measurement of the entire house. Recent advances in deep learning gave rise to many methods that mostly focus on learning a direct mapping from aggregate measurement to individual appliance power. However, these methods are not only computationally expensive, but they often suffer from overfitting and do not generalize very well. In this article, we propose a novel NILM method that leverages advances in statistical learning that have not been properly applied in this domain before. The proposed method consists of three stages: first, a Bayesian nonparametric learning-based approach for appliance state extraction; second, synthetic minority oversampling technique for data augmentation and mitigating the heavy imbalance in switching events; and third, appliance-specific lightweight long short-term memory networks for status classification for each appliance. We adopt a “differential” input (the difference before and after the switching event) to reduce the complexity of network training and make the proposed method robust to multiappliance switching events. Experiments are conducted to demonstrate the effectiveness of the proposed method, achieving superior performance when compared to recent methods. An ablation study is conducted to demonstrate the effectiveness of each module of our method. Finally, we investigate the quality of generated synthetic samples.

Forecasting with Bayesian Grouped Random Effects in Panel Data

In this paper, we estimate and leverage latent constant group structure to generate the point, set, and density forecasts for short dynamic panel data. We implement a nonparametric Bayesian approach to simultaneously identify coefficients and group membership in the random effects which are heterogeneous across groups but fixed within a group. This method allows us to flexibly incorporate subjective prior knowledge on the group structure that potentially improves the predictive accuracy. In Monte Carlo experiments, we demonstrate that our Bayesian grouped random effects (BGRE) estimators produce accurate estimates and score predictive gains over standard panel data estimators. With a data-driven group structure, the BGRE estimators exhibit comparable accuracy of clustering with the Kmeans algorithm and outperform a two-step Bayesian grouped estimator whose group structure relies on Kmeans. In the empirical analysis, we apply our method to forecast the investment rate across a broad range of firms and illustrate that the estimated latent group structure facilitates forecasts relative to standard panel data estimators.

Bernstein--von Mises Theorems and Uncertainty Quantification for Linear Inverse Problems

We consider the statistical inverse problem of recovering an unknown function $f$ from a linear measurement corrupted by additive Gaussian white noise. We employ a nonparametric Bayesian approach with standard Gaussian priors, for which the posterior-based reconstruction of $f$ corresponds to a Tikhonov regularizer $\bar f$ with a reproducing kernel Hilbert space norm penalty. We prove a semiparametric Bernstein--von Mises theorem for a large collection of linear functionals of $f$, implying that semiparametric posterior estimation and uncertainty quantification are valid and optimal from a frequentist point of view. The result is applied to study three concrete examples that cover both the mildly and severely ill-posed cases: specifically, an elliptic inverse problem, an elliptic boundary value problem, and the heat equation. For the elliptic boundary value problem, we also obtain a nonparametric version of the theorem that entails the convergence of the posterior distribution to a prior-independent infinite-dimensional Gaussian probability measure with minimal covariance. As a consequence, it follows that the Tikhonov regularizer $\bar f$ is an efficient estimator of $f$, and we derive frequentist guarantees for certain credible balls centered at $\bar{f}$.

Nonparametric prior elicitation for a binomial proportion

This paper proposes a nonparametric Bayesian approach based on a density estimation with an open unit interval (0,1) using binomial data. We propose a very efficient nonparametric Bayesian approach method to infer smooth density defined on (0,1) through the transformation of a random variable. For practical implementation, we provide the corresponding blocked Gibbs sampling procedure based on the stick-breaking representation. The greatest advantage of this method is that it does not require us to draw from the complete conditional posterior distribution using a Metropolis-Hastings transition probability because the proposed transformation leads to a pair of conjugate priors and likelihoods. The validity of the proposed method is assessed through simulated and real data analysis.

A hierarchical dependent Dirichlet process prior for modelling bird migration patterns in the UK

Environmental changes in recent years have been linked to phenological shifts which in turn are linked to the survival of species. The work in this paper is motivated by capture-recapture data on blackcaps collected by the British Trust for Ornithology as part of the Constant Effort Sites monitoring scheme. Blackcaps overwinter abroad and migrate to the UK annually for breeding purposes. We propose a novel Bayesian nonparametric approach for expressing the bivariate density of individual arrival and departure times at different sites across a number of years as a mixture model. The new model combines the ideas of the hierarchical and the dependent Dirichlet process, allowing the estimation of site-specific weights and year-specific mixture locations, which are modelled as functions of environmental covariates using a multivariate extension of the Gaussian process. The proposed modelling framework is extremely general and can be used in any context where multivariate density estimation is performed jointly across different groups and in the presence of a continuous covariate.