Conditional Posterior Density Research Articles

Gibbs sampler for Bayesian prediction of triple seasonal autoregressive processes

Researchers have extended autoregressive (AR) time-series models to adequately fit and model time-series with triple seasonality. These AR extensions can be referred to as triple seasonal AR (TSAR) models. For these TSAR time-series models, only Bayesian estimation and identification have been introduced. Therefore, in this article, we aim to extend the existing work for presenting the Bayesian prediction for TSAR models using the Gibbs sampler algorithm. In this Bayesian prediction, we first assume the TSAR errors are identically normally distributed, and we employ normal-gamma and g priors for the TSAR parameters. Based on the normally distributed TSAR errors and the specified TSAR parameters’ priors, we are able to derive the conditional predictive and posterior densities. Particularly, we show the conditional posterior densities of TSAR coefficients and variance are the multivariate normal and inverse-gamma, respectively. In addition, for the future TSAR observations, we show the conditional predictive density is the multivariate normal. Using these derived full conditional distributions, we propose the Gibbs sampler to efficiently approximate the joint predictive and posterior densities and to easily carry out multiple-step ahead predictions. We validate the accuracy of forecasting for the proposed Gibbs sampler by conducting a simulation study. Moreover, we apply the proposed Gibbs sampler to hourly electricity-load time-series datasets in some European countries, along with a comparison with the well-known long-short term memory (LSTM) recurrent neural model.

Photoacoustic quantification of tissue oxygenation using conditional invertible neural networks.

Intelligent systems in interventional healthcare depend on the reliable perception of the environment. In this context, photoacoustic tomography (PAT) has emerged as a non-invasive, functional imaging modality with great clinical potential. Current research focuses on converting the high-dimensional, not human-interpretable spectral data into the underlying functional information, specifically the blood oxygenation. One of the largely unexplored issues stalling clinical advances is the fact that the quantification problem is ambiguous, i.e. that radically different tissue parameter configurations could lead to almost identical photoacoustic spectra. In the present work, we tackle this problem with conditional Invertible Neural Networks (cINNs). Going beyond traditional point estimates, our network is used to compute an approximation of the conditional posterior density of tissue parameters given the measurement. To this end, an automatic mode detection algorithm extracts the plausible solution from the sample-based posterior. According to a comprehensive validation study based on both synthetic and real images, our approach is well-suited for exploring ambiguity in quantitative PAT.

Bayesian uncertainty quantification for guided-wave-based multidamage localization in plate-like structures using Gibbs sampling

In this article, a new Bayesian approach for guided-wave-based multidamage localization by employing Gibbs sampling is proposed. By using the information of time-of-flight (ToF) embedded in guided wave signals, the posterior probability distributions of three parameter groups, that is, the horizontal and vertical coordinates of the multidamage locations (x, y) and wave velocity v, are characterized using Gibbs sampling samples. To obtain the analytical form of the conditional posterior probability density function of each parameter group conditional on the other two and the available ToF data, a first-order Taylor expansion of the nonlinear ToF-based damage localization model with respect to each parameter group is performed. Two Gibbs sampling algorithms are proposed, which differ in their strategies to address the posterior uncertainty of the prediction error parameter; however, both algorithms iteratively sample from conditional posterior probability density functions of three parameter groups. Therefore, the effective number of dimensions for Gibbs sampling is always three, regardless of the number of defects. The final damage localization results are obtained by grouping all ToFs and then comparing the posterior uncertainty of localization results of each grouping scheme to obtain the most reliable sampling results among all candidates. The proposed method not only identifies the group velocity but also localizes multiple defects by sharing the same characteristics of damage localization. Furthermore, this method can quantify the uncertainty of multidamage localization to automatically find the most reliable damage locations. The effectiveness and robustness of the proposed algorithms are validated by both numerical and experimental examples.

Bayesian Estimation of an M/M/𝑅 Queue with Heterogeneous Servers Using Markov Chain Monte Carlo Method

AbstractIn this paper, we consider the Bayesian inference of M/M/𝑅 queue with 𝑅 heterogeneous servers with service rates \mu_{1},\mu_{2},\ldots,\mu_{R}, where \mu_{1}>\mu_{2}>\cdots>\mu_{R}. Assuming multivariate gamma prior distribution for service rates and gamma prior distribution for arrival rate 𝜆, we derive the conditional posterior densities of mean arrival rate and mean service rates. We apply the Markov chain Monte Carlo method and compute the Bayes estimates and credible interval for the M/M/3 queue, as a particular case of the M/M/𝑅 queue under squared error loss function, entropy loss function and linex loss function corresponding to a different set of hyperparameters.

Bayesian Estimate of Parameters for ARMA Model Forecasting

Abstract This paper presents a Bayesian approach to finding the Bayes estimator of parameters for ARMA model forecasting under normal-gamma prior assumption with a quadratic loss function in mathematical expression. Obtaining the conditional posterior predictive density is based on the normal-gamma prior and the conditional predictive density, whereas its marginal conditional posterior predictive density is obtained using the conditional posterior predictive density. Furthermore, the Bayes estimator of parameters is derived from the marginal conditional posterior predictive density.

An MCMC computational approach for a continuous time state-dependent regime switching diffusion process

ABSTRACT State-dependent regime switching diffusion processes or hybrid switching diffusion (HSD) processes are hard to simulate with classical methods which leads us to adopt a Markov chain Monte Carlo (MCMC) Bayesian approach very convenient to estimate complicated models such as the HSD one. In the HSD, the diffusion component is dependent on the switching discrete hidden regimes and the transition rates of the regime switching are dependent on the diffusion observations. Since in reality phenomena are only observed in discrete times, data imputation is called for to create more observations so as to have good approximations for the density of the diffusion process. Three categories of entities will be computed in a Bayesian context: The latent imputed observations, the regime switching states, and the parameters of the models. The latent imputed data is updated at random time intervals in block using a Metropolis Hastings algorithm. The switching states are computed by an adaptation of a forward filtering backward smoothing algorithm to the HSD model. The parameters are estimated after prior specifications and conditional posterior densities formulation using Gibbs sampler or Metropolis Hastings algorithm.

Partition Weighted Approach For Estimating the Marginal Posterior Density With Applications

The computation of marginal posterior density in Bayesian analysis is essential in that it can provide complete information about parameters of interest. Furthermore, the marginal posterior density can be used for computing Bayes factors, posterior model probabilities, and diagnostic measures. The conditional marginal density estimator (CMDE) is theoretically the best for marginal density estimation but requires the closed-form expression of the conditional posterior density, which is often not available in many applications. We develop the partition weighted marginal density estimator (PWMDE) to realize the CMDE. This unbiased estimator requires only a single Markov chain Monte Carlo output from the joint posterior distribution and the known unnormalized posterior density. The theoretical properties and various applications of the PWMDE are examined in detail. The PWMDE method is also extended to the estimation of conditional posterior densities. We carry out simulation studies to investigate the empirical performance of the PWMDE and further demonstrate the desirable features of the proposed method with two real data sets from a study of dissociative identity disorder patients and a prostate cancer study, respectively. Supplementary materials for this article are available online.

Bayesian inference on contingency tables with uncertainty about independence for small areas

ABSTRACTA scientist might have vague information about independence/dependence in a two-way table, and a statistician might proceed with estimation conditional on this piece of information. However, one needs to take into account the uncertainty in this information which can increase variability. We develop a Bayesian method to solve this problem when estimation is needed for the cells of a contingency table and there is uncertainty about independence or dependence. In our problem, there are several small areas and a table is constructed for each area. We use the hierarchical Dirichlet-multinomial model to analyze the counts from these small areas. The key idea in our method is that the cell probabilities of each area is expressed as a convex combination of the cell probabilities under independence and the cell probabilities under dependence, where each area has its own unknown weight. We show how to fit the model using the Gibbs sampler even though many of the conditional posterior densities are nonstandard. As a by product of our method, we have actually produced a test of independence which is competitive to the chi-square test for a single table. To illustrate our method, we have used an example on body mass index and bone mineral density data obtained from NHANES III. We have shown some important differences among the three scenarios (independence, dependence and the convex combination of these two) when Bayesian predictive inference is done for the finite population means corresponding to each cell of the table.

A Simple Analysis of the Exact Probability Matching Prior in the Location-Scale Model

ABSTRACTIt has long been asserted that in univariate location-scale models, when concerned with inference for either the location or scale parameter, the use of the inverse of the scale parameter as a Bayesian prior yields posterior credible sets that have exactly the correct frequentist confidence set interpretation. This claim dates to at least Peers, and has subsequently been noted by various authors, with varying degrees of justification. We present a simple, direct demonstration of the exact matching property of the posterior credible sets derived under use of this prior in the univariate location-scale model. This is done by establishing an equivalence between the conditional frequentist and posterior densities of the pivotal quantities on which conditional frequentist inferences are based.

Bayesian tobit quantile regression with penalty

ABSTRACTTobit quantile regression (QReg) model provides an efficient way of coping with left-censored data and can be viewed as a linear QReg model, where only the data on the dependent variable is incompletely observed. This article considers the regularizer in tobit QReg from a Bayesian perspective. By proposing a hierarchical model framework, we consider a fully Bayesian treatment that leads to a Gibbs sampling algorithm with fully tractable conditional posterior densities. The proposed method is illustrated via simulation studies and a real dataset. Results show that the proposed method performs very well in comparison to other approaches.

Signal characterization using Hidden Markov Models with applications in acoustical oceanography

The work presents a method for characterizing underwater acoustic signals using a Markov chain, with hidden variables, based on their wavelet transform. Initially, we assign to the signal a Hidden Markov Model (HMM) for which the conditional posterior probability density function seems to be the most representative using an Expectation-Maximization algorithm. Special techniques are applied to avoid over-fitting which in principle is not desirable for the sought applications. The features used for the assignment consist of two dimensional time series obtained by preprocessing of signal’s wavelet packet coefficients. Subsequently, we use an approximation of the Kullback Leibler (KL) divergence as a similarity measure among the HMMs. The approximation is obtained by employing Monte-Carlo (MC) techniques simulating the significant sampling from the HMMs posterior distributions. This technique is used in cases where the similarity of two or more signals is to be exploited. These cases include a variety of problems associated with the monitoring of the marine environment using acoustic or seismic signals. The applications to be presented here are referred to problems of geoacoustic inversions (seabed mapping) using simulated acoustic data and seismic monitoring using real data from a terrestrial seismograph to illustrate the various possible applications of the suggested method.

Bayesian system identification based on hierarchical sparse Bayesian learning and Gibbs sampling with application to structural damage assessment

Bayesian system identification has attracted substantial interest in recent years for inferring structural models based on measured dynamic response from a structural dynamical system. The focus in this paper is Bayesian system identification based on noisy incomplete modal data where we can impose spatially-sparse stiffness changes when updating a structural model. To this end, based on a similar hierarchical sparse Bayesian learning model from our previous work, we propose two Gibbs sampling algorithms. The algorithms differ in their strategies to deal with the posterior uncertainty of the equation-error precision parameter, but both sample from the conditional posterior probability density functions (PDFs) for the structural stiffness parameters and system modal parameters. The effective dimension for the Gibbs sampling is low because iterative sampling is done from only three conditional posterior PDFs that correspond to three parameter groups, along with sampling of the equation-error precision parameter from another conditional posterior PDF in one of the algorithms where it is not integrated out as a “nuisance” parameter. A nice feature from a computational perspective is that it is not necessary to solve a nonlinear eigenvalue problem of a structural model. The effectiveness and robustness of the proposed algorithms are illustrated by applying them to the IASE-ASCE Phase II simulated and experimental benchmark studies. The goal is to use incomplete modal data identified before and after possible damage to detect and assess spatially-sparse stiffness reductions induced by any damage. Our past and current focus on meeting challenges arising from Bayesian inference of structural stiffness serves to strengthen the capability of vibration-based structural system identification but our methods also have much broader applicability for inverse problems in science and technology where system matrices are to be inferred from noisy partial information about their eigenquantities.

Bayesian Inference for State Space Model with Panel Data

Abstract The present work explores panel data set-up in a Bayesian state space model. The conditional posterior densities of parameters are utilized to determine the marginal posterior densities using the Gibbs sampler. An efficient one step ahead predictive density mechanism is developed to further the state of art in prediction-based decision making.

Bayesian inference for state space model with panel data

The present work explores panel data set-up in a Bayesian state space model. The conditional posterior densities of parameters are utilized to determine the marginal posterior densities using the Gibbs sampler. An efficient one step ahead predictive density mechanism is developed to further the state of art in prediction based decision making.

An Efficient Independence Sampler for Updating Branches in Bayesian Markov chain Monte Carlo Sampling of Phylogenetic Trees.

Sampling tree space is the most challenging aspect of Bayesian phylogenetic inference. The sheer number of alternative topologies is problematic by itself. In addition, the complex dependency between branch lengths and topology increases the difficulty of moving efficiently among topologies. Current tree proposals are fast but sample new trees using primitive transformations or re-mappings of old branch lengths. This reduces acceptance rates and presumably slows down convergence and mixing. Here, we explore branch proposals that do not rely on old branch lengths but instead are based on approximations of the conditional posterior. Using a diverse set of empirical data sets, we show that most conditional branch posteriors can be accurately approximated via a distribution. We empirically determine the relationship between the logarithmic conditional posterior density, its derivatives, and the characteristics of the branch posterior. We use these relationships to derive an independence sampler for proposing branches with an acceptance ratio of ∼90% on most data sets. This proposal samples branches between 2× and 3× more efficiently than traditional proposals with respect to the effective sample size per unit of runtime. We also compare the performance of standard topology proposals with hybrid proposals that use the new independence sampler to update those branches that are most affected by the topological change. Our results show that hybrid proposals can sometimes noticeably decrease the number of generations necessary for topological convergence. Inconsistent performance gains indicate that branch updates are not the limiting factor in improving topological convergence for the currently employed set of proposals. However, our independence sampler might be essential for the construction of novel tree proposals that apply more radical topology changes.

Posterior Analysis of State Space Model with Spherical Symmetricity

The present work investigates state space model with nonnormal disturbances when the deviation from normality has been observed only with respect to kurtosis and the distribution of disturbances continues to follow a symmetric family of distributions. Spherically symmetric distribution is used to approximate behavior of symmetric nonnormal disturbances for discrete time series. The conditional posterior densities of the involved parameters are derived, which are further utilized in Gibbs sampler scheme for estimating the marginal posterior densities. The state space model with disturbances following multivariate-tdistribution, which is a particular case of spherically symmetric distribution, is discussed.

Inversion and uncertainty of highly parameterized models in a Bayesian framework by sampling the maximal conditional posterior distribution of parameters

We introduce the concept of maximal conditional posterior distribution (MCPD) to assess the uncertainty of model parameters in a Bayesian framework. Although, Markov Chains Monte Carlo (MCMC) methods are particularly suited for this task, they become challenging with highly parameterized nonlinear models. The MCPD represents the conditional probability distribution function of a given parameter knowing that the other parameters maximize the conditional posterior density function. Unlike MCMC which accepts or rejects solutions sampled in the parameter space, MCPD is calculated through several optimization processes. Model inversion using MCPD algorithm is particularly useful for highly parameterized problems because calculations are independent. Consequently, they can be evaluated simultaneously with a multi-core computer. In the present work, the MCPD approach is applied to invert a 2D stochastic groundwater flow problem where the log-transmissivity field of the medium is inferred from scarce and noisy data. For this purpose, the stochastic field is expanded onto a set of orthogonal functions using a Karhunen–Loève (KL) transformation. Though the prior guess on the stochastic structure (covariance) of the transmissivity field is erroneous, the MCPD inference of the KL coefficients is able to extract relevant inverse solutions.

A hierarchical conditional autoregressive model for colorectal cancer survival data

In this article, we propose a Bayesian hierarchical linear mixed model to cancer data with geographical characteristics. The spatial effects are captured via a conditional autoregressive (CAR) model. The survival model is introduced to analyze the survival pattern of colorectal cancer based on geographical factors and patients' disease conditions. We propose a special case of the classic Cox model with Weibull hazard as well as a cure rate model. A CAR prior is used to capture the spatial effects, which are determined based on counties the survival subjects belong to. The computation is done via Gibbs sampling. The ratio‐of‐uniforms method is used to sample from a nonstandard conditional posterior density, and the adaptive rejection method is used to sample from log‐concave densities. The model is applied to the Colon & Rectum Cancer incidences in Iowa from the Surveillance, Epidemiology, and End Results (SEER) database. Computation is implemented in Intel Fortran on Linux platform. WIREs Comput Stat 2014, 6:37–44. doi: 10.1002/wics.1283This article is categorized under: Statistical and Graphical Methods of Data Analysis > Markov Chain Monte Carlo (MCMC) Statistical and Graphical Methods of Data Analysis > Reliability, Survivability, and Quality Control

Methods for computing marginal data densities from the Gibbs output

We introduce two estimators for estimating the Marginal Data Density (MDD) from the Gibbs output. Our methods are based on exploiting the analytical tractability condition, which requires that some parameter blocks can be analytically integrated out from the conditional posterior densities. This condition is satisfied by several widely used time series models. An empirical application to six-variate VAR models shows that the bias of a fully computational estimator is sufficiently large to distort the implied model rankings. One of the estimators is fast enough to make multiple computations of MDDs in densely parameterized models feasible.

Back to Basics for Bayesian Model Building in Genomic Selection

Numerous Bayesian methods of phenotype prediction and genomic breeding value estimation based on multilocus association models have been proposed. Computationally the methods have been based either on Markov chain Monte Carlo or on faster maximum a posteriori estimation. The demand for more accurate and more efficient estimation has led to the rapid emergence of workable methods, unfortunately at the expense of well-defined principles for Bayesian model building. In this article we go back to the basics and build a Bayesian multilocus association model for quantitative and binary traits with carefully defined hierarchical parameterization of Student's t and Laplace priors. In this treatment we consider alternative model structures, using indicator variables and polygenic terms. We make the most of the conjugate analysis, enabled by the hierarchical formulation of the prior densities, by deriving the fully conditional posterior densities of the parameters and using the acquired known distributions in building fast generalized expectation-maximization estimation algorithms.