Full Conditional Distributions Research Articles

A general Bayesian model for heteroskedastic data with fully conjugate full-conditional distributions

Models for heteroskedastic data are relevant in a variety of applications ranging from financial time series to environmental statistics. However, the topic of modelling the variance function conditionally has not seen as much attention as modelling the mean. Volatility models have been used in specific applications, but these models can be difficult to fit in a Bayesian setting due to posterior distributions that are challenging to sample efficiently. In this work, we introduce a general model for heteroskedastic data. This approach models the conditional variance as a function of any desired covariates or random effects. We rely on multivariate log-Gamma distribution theory to construct priors that yield fully conjugate full-conditional distributions for Gibbs sampling. Furthermore, we extend the model to a deep learning approach that can provide highly accurate estimates for time dependent data. We also provide an extension for heavy-tailed data. We illustrate our methodology via three applications.

A Gibbs sampler for the multidimensional four-parameter logistic item response model via a data augmentation scheme.

The four-parameter logistic (4PL) item response model, which includes an upper asymptote for the correct response probability, has drawn increasing interest due to its suitability for many practical scenarios. This paper proposes a new Gibbs sampling algorithm for estimation of the multidimensional 4PL model based on an efficient data augmentation scheme (DAGS). With the introduction of three continuous latent variables, the full conditional distributions are tractable, allowing easy implementation of a Gibbs sampler. Simulation studies are conducted to evaluate the proposed method and several popular alternatives. An empirical data set was analysed using the 4PL model to show its improved performance over the three-parameter and two-parameter logistic models. The proposed estimation scheme is easily accessible to practitioners through the open-source IRTlogit package.

Full conditional distributions for Bayesian multilevel models with additive or interactive effects and missing data on covariates

Missing data are a common occurrence in analyses of multivariate data, including in multilevel modeling. Bayesian approaches to handling missing data in multilevel modeling have garnered increasing attention, either on their own or in service of multiple imputation. However, these applications are largely confined to specific models or missingness patterns. The current work provides a coherent account of Bayesian analysis of multilevel models in the presence of missing data on the outcomes, level-1 predictors, and level-2 predictors, that covers the main aspects of the models and missingness. In doing so, this work provides a grounding for estimation in fully Bayesian approaches that employ Gibbs sampling, and provides an account of how to generate the imputations in the first phase of a multiple imputation approach.

Bayesian semiparametric modeling of response mechanism for nonignorable missing data

Statistical inference with nonresponse is quite challenging, especially when the response mechanism is nonignorable. In this case, the validity of statistical inference depends on untestable correct specification of the response model. To avoid the misspecification, we propose semiparametric Bayesian estimation in which an outcome model is parametric, but the response model is semiparametric in that we do not assume any parametric form for the nonresponse variable. We adopt penalized spline methods to estimate the unknown function. We also consider a fully nonparametric approach to modeling the response mechanism by using radial basis function methods. Using Pólya–gamma data augmentation, we developed an efficient posterior computation algorithm via Gibbs sampling in which most full conditional distributions can be obtained in familiar forms. The performance of the proposed method is demonstrated in simulation studies and an application to longitudinal data.

Fast Bayesian inference using Laplace approximations in nonparametric double additive location-scale models with right- and interval-censored data

Penalized B-splines are commonly used in additive models to describe smooth changes in a response with quantitative covariates. This is usually done through the conditional mean in the exponential family using generalized additive models with an indirect impact on other conditional moments. Another common strategy is to focus on several low-order conditional moments, leaving the full conditional distribution unspecified. Alternatively, a multi-parameter distribution could be assumed for the response with several of its parameters jointly regressed on covariates using additive expressions. The latter proposal for a right- or interval-censored continuous response with a highly flexible and smooth nonparametric density is considered. The focus is on location-scale models with additive terms in the conditional mean and standard deviation. Starting from recent results in the Bayesian framework, a fast converging algorithm is proposed to select penalty parameters from their marginal posteriors. It is based on Laplace approximations of the conditional posterior of the spline parameters. Simulations suggest that the estimators obtained in this way have excellent frequentist properties and superior efficiencies compared to approaches with a working Gaussian assumption. The methodology is illustrated by the analysis of right- and interval-censored income data.

Deep distribution regression

Due to their flexibility and predictive performance, machine-learning based regression methods have become an important tool for predictive modeling and forecasting. However, most methods focus on estimating the conditional mean or specific quantiles of the target quantity and do not provide the full conditional distribution, which contains uncertainty information that might be crucial for decision making. A general solution consists of transforming a conditional distribution estimation problem into a constrained multi-class classification problem, in which tools such as deep neural networks can be applied. A novel joint binary cross-entropy loss function is proposed to accomplish this goal. Its performance is compared to current state-of-the-art methods via simulation. The approach also shows improved accuracy in a probabilistic solar energy forecasting problem.

Gibbs Flow for Approximate Transport with Applications to Bayesian Computation

AbstractLet π0 and π1 be two distributions on the Borel space (Rd,B(Rd)). Any measurable function T:Rd→Rd such that Y=T(X)∼π1 if X∼π0 is called a transport map from π0 to π1. For any π0 and π1, if one could obtain an analytical expression for a transport map from π0 to π1, then this could be straightforwardly applied to sample from any distribution. One would map draws from an easy-to-sample distribution π0 to the target distribution π1 using this transport map. Although it is usually impossible to obtain an explicit transport map for complex target distributions, we show here how to build a tractable approximation of a novel transport map. This is achieved by moving samples from π0 using an ordinary differential equation with a velocity field that depends on the full conditional distributions of the target. Even when this ordinary differential equation is time-discretised and the full conditional distributions are numerically approximated, the resulting distribution of mapped samples can be efficiently evaluated and used as a proposal within sequential Monte Carlo samplers. We demonstrate significant gains over state-of-the-art sequential Monte Carlo samplers at a fixed computational complexity on a variety of applications.

Bayesian Spatial Split-Population Models for the Social Sciences

Survival data often include an “immune” or cured fraction of units that will never experience an event and conversely, an “at risk” fraction that can fail or die. It is also plausible that spatial clustering (i.e., spatial autocorrelation) in latent or unmeasured risk factors among adjacent units can affect their odds of being immune and survival time of interest. To address these methodological challenges, this article introduces a class of parametric Spatial split-population survival models—also denoted as Spatial cure models—that explicitly accounts for the influence of spatial autocorrelation among the underlying risk propensities of units on not only their probability of being immune but also their risk of experiencing the event of interest. Our approach is Bayesian in that we account for spatial autocorrelation in unmeasured risk factors across adjacent units in the cure model’s split-stage (cure rate portion) and survival stage via the conditional autoregressive prior (CAR) prior. The article also presents a set of parametric split-population survival models with non-spatial i.i.d frailties and without frailties, and time-varying covariates can be included in all the models mentioned above. Bayesian inference of the non-spatial and spatial cure models is conducted via a hybrid Markov Chain Monte Carlo (MCMC) algorithm. The relevant full conditional distributions required for MCMC sampling are also derived and presented in the paper. We fit all the models to survival data on post-civil war peace to demonstrate their main applicability and main features.

Bayesian Study Using MCMC of Three-Parameter Frechet Distribution Based on Type-I Censored Data

Type-I censoring mechanism arises when the number of units experiencing the event is random but the total duration of the study is fixed. There are a number of mathematical approaches developed to handle this type of data. The purpose of the research was to estimate the three parameters of the Frechet distribution via the frequentist Maximum Likelihood and the Bayesian Estimators. In this paper, the maximum likelihood method (MLE) is not available of the three parameters in the closed forms; therefore, it was solved by the numerical methods. Similarly, the Bayesian estimators are implemented using Jeffreys and gamma priors with two loss functions, which are: squared error loss function and Linear Exponential Loss Function (LINEX). The parameters of the Frechet distribution via Bayesian cannot be obtained analytically and therefore Markov Chain Monte Carlo is used, where the full conditional distribution for the three parameters is obtained via Metropolis-Hastings algorithm. Comparisons of the estimators are obtained using Mean Square Errors (MSE) to determine the best estimator of the three parameters of the Frechet distribution. The results show that the Bayesian estimation under Linear Exponential Loss Function based on Type-I censored data is a better estimator for all the parameter estimates when the value of the loss parameter is positive.

Bayesian estimation and model comparison for linear dynamic panel models with missing values

SummaryPanel data are collected over several time periods for the same units and hence allow for modelling both latent heterogeneity and dynamics. Since in a dynamic setup, the dependent variable also appears as an explanatory variable in later periods, missing values lead to substantial loss of information and the possibility of inefficient estimation. For linear dynamic panel models with fixed or random effects, we suggest a Bayesian approach to deal with missing values. The Gibbs sampling scheme providing a sample from the posterior distribution is thereby augmented by draws from the full conditional distribution of the missing values. While the full conditional distribution for missing values in the dependent variable is implied by the model setup, we incorporate a flexible non‐parametric approximation to the full conditional posterior distribution of missing values in the explaining variables. Also, we provide accurate non‐nested model comparison in terms of the marginal likelihood from the resulting hybrid Gibbs sampling output. The properties and possible efficiency gains of the suggested approach are illustrated by means of a simulation study and an empirical application using a macroeconomic panel data set.

Synthetic Microdata for Establishment Surveys Under Informative Sampling

AbstractMany agencies are investigating whether releasing synthetic microdata could be a viable dissemination strategy for highly sensitive data, such as business data, for which disclosure avoidance regulations otherwise prohibit the release of public use microdata. However, existing methods assume that the original data either cover the entire population or comprise a simple random sample, which limits the application of these methods in the context of survey data with unequal weights. This paper discusses synthetic data generation under informative sampling. To utilise design information in survey weights, we rely on the pseudo likelihood approach when building a hierarchical Bayesian model to estimate the distribution of the finite population. Then, synthetic populations are randomly drawn from the estimated finite population density. We present the full conditional distributions of the Markov chain Monte Carlo algorithm for posterior inference with the pseudo likelihood function. Using simulation studies, we show that the suggested synthetic data approach offers high utility for design-based and model-based analyses while offering a high level of disclosure protection. We apply the proposed method to a subset of the 2012 U.S. Economic Census and evaluate results with utility metrics and disclosure avoidance metrics under data attacker scenarios commonly used for business data.

Bayesian estimation for threshold autoregressive model with multiple structural breaks

This paper provides a Bayesian setup for multiple regimes threshold autoregressive model with possible break points. A full conditional posterior distribution is obtained for all model parameters with considering suitable prior information. Threshold and break point variables do not attain standard form distributions. To compute posterior distributions, we apply the Gibbs sampler with the Metropolis-Hastings algorithm. A variety of loss functions are considered for optimizing the risk associated with each parameter. For empirical evidence, simulation study and real data illustration are carried out.

Clustering for time-varying relational count data

Relational count data are often obtained from sources such as simultaneous purchase in online shops and social networking service information. Clustering such relational count data reveals the latent structure of the relationship between objects such as household items or people. When relational count data observed at multiple time points are available, it is worthwhile incorporating the time structure into the clustering result to understand how objects move between the clusters over time. In this paper, we propose two clustering methods for analyzing time-varying relational count data. The first model, the dynamic Poisson infinite relational model (dPIRM), handles time-varying relational count data. In the second model, which we call the dynamic zero-inflated Poisson infinite relational model, we further extend the dPIRM so that it can handle zero-inflated data. Proposing both two models is important as zero-inflated data are often encountered, especially when the time intervals are short. In addition, by explicitly deriving the relevant full conditional distributions, we describe the features of the estimated parameters and, in turn, the relationship between the two models. We show the effectiveness of both models through a simulation study and a real data example.

Contrast trees and distribution boosting

A method for decision tree induction is presented. Given a set of predictor variables [Formula: see text] and two outcome variables y and z associated with each x, the goal is to identify those values of x for which the respective distributions of [Formula: see text] and [Formula: see text], or selected properties of those distributions such as means or quantiles, are most different. Contrast trees provide a lack-of-fit measure for statistical models of such statistics, or for the complete conditional distribution [Formula: see text], as a function of x. They are easily interpreted and can be used as diagnostic tools to reveal and then understand the inaccuracies of models produced by any learning method. A corresponding contrast-boosting strategy is described for remedying any uncovered errors, thereby producing potentially more accurate predictions. This leads to a distribution-boosting strategy for directly estimating the full conditional distribution of y at each x under no assumptions concerning its shape, form, or parametric representation.

Bayesian Change Point Estimation Based on Masked Data in Exponential Distribution Parallel System

Change point reflects a qualitative change in things. It has gained some applications in the field of reliability. In order to estimate the position parameters of the change point, a Bayesian change point model based on masked data and Gibbs sampling was proposed. By filling in missing lifetime data and introducing latent variables, the simple likelihood function is obtained for exponential distribution parallel system under censored data. This paper describes the probability distributions and random generation methods of the missing lifetime variables and latent variables, and obtains the full conditional distributions of the change point position parameters and other unknown parameters. By Gibbs sampling and estimation of unknown parameters, the estimates of the mean, median, and quantile of the parameter posterior distribution are obtained. The specific steps of Gibbs sampling are introduced in detail. The convergence of Gibbs sampling is also diagnosed. Random simulation results show that the estimations are fairly accurate.

A Bayesian approach to disease clustering using restricted Chinese restaurant processes

Identifying disease clusters (areas with an unusually high incidence of a particular disease) is a common problem in epidemiology and public health. We describe a Bayesian nonparametric mixture model for disease clustering that constrains clusters to be made of adjacent areal units. This is achieved by modifying the exchangeable partition probability function associated with the Ewen’s sampling distribution. We call the resulting prior the Restricted Chinese Restaurant Process, as the associated full conditional distributions resemble those associated with the standard Chinese Restaurant Process. The model is illustrated using synthetic data sets and in an application to oral cancer mortality in Germany.

Conjugate Bayesian unit‐level modelling of count data under informative sampling designs

Unit‐level models for survey data offer many advantages over their area‐level counterparts, such as potential for more precise estimates and a natural benchmarking property. However, two main challenges occur in this context: accounting for an informative survey design and handling non‐Gaussian data types. The pseudo‐likelihood approach is one solution to the former, and conjugate multivariate distribution theory offers a solution to the latter. By combining these approaches, we attain a unit‐level model for count data that accounts for informative sampling designs and includes a fully Bayesian model uncertainty propagation. Importantly, conjugate full conditional distributions hold under the pseudo‐likelihood, yielding an extremely computationally efficient approach. Our method is illustrated via an empirical simulation study using count data from the American Community Survey public use microdata sample.

Gibbs sampling using the data augmentation scheme for higher-order item response models

Many latent traits in the human sciences have a hierarchical structure. This article will focus on a higher-order item response theory ( HO-IRT) model, which integrates a single overall ability and several domain-specific abilities in the same model to improve the parameter estimation of assessment data. A DAGS-based (data augmentation scheme) Gibbs sampler procedure to analyze HO-IRT models with three-parameter logistic link will be introduced. This procedure is a generalization of Maris and Maris (2002)’s sampling based Bayesian technique, called the DA-T-Gibbs sampler, are suitable for a wide variety of IRT models. With the introduction of the two latent variables, the full conditional distributions are tractable, allowing easy implementation of a Gibbs sampler. The performance of the proposed DAGS-based Bayesian procedure is evaluated via a simulation study and compared with the M–H algorithm. Results indicate that the proposed DAGS-based Bayesian procedure is more efficient and flexible than the M–H algorithm. Finally, applications to a real dataset are conducted to demonstrate the efficiency and utility of the proposed method.

Bayesian Hierarchical Models With Conjugate Full-Conditional Distributions for Dependent Data From the Natural Exponential Family

We introduce a Bayesian approach for analyzing (possibly) high-dimensional dependent data that are distributed according to a member from the natural exponential family of distributions. This problem requires extensive methodological advancements, as jointly modeling high-dimensional dependent data leads to the so-called “big n problem.” The computational complexity of the “big n problem” is further exacerbated when allowing for non-Gaussian data models, as is the case here. Thus, we develop new computationally efficient distribution theory for this setting. In particular, we introduce the “conjugate multivariate distribution,” which is motivated by the Diaconis and Ylvisaker distribution. Furthermore, we provide substantial theoretical and methodological development including: results regarding conditional distributions, an asymptotic relationship with the multivariate normal distribution, conjugate prior distributions, and full-conditional distributions for a Gibbs sampler. To demonstrate the wide-applicability of the proposed methodology, we provide two simulation studies and three applications based on an epidemiology dataset, a federal statistics dataset, and an environmental dataset, respectively. Supplementary materials for this article are available online.

Bayesian approach to the zinc extraction curve of soil with sewage sludge

Zinc uptake is essential for crop development; thus, knowledge about soil zinc availability is fundamental for fertilization in periods of higher crop demand. A nonlinear first-order kinetic model has been employed to evaluate zinc availability. Studies usually employ few observations; however, inference in nonlinear models is only valid for sufficiently large samples. An alternative is the Bayesian method, where inferences are made in terms of probability, which is effective even with small samples. The aim of this study was to use Bayesian methodology to evaluate the fitness of a nonlinear first-order kinetic model to describe zinc extraction from soil with sewage sludge using seven different extraction solutions. The analysed data were obtained from an experiment using a completely randomized design and three replicates. Fifteen zinc extractions were evaluated for each extraction solution. Posterior distributions of a study that evaluated the nonlinear first-order kinetic model were used as prior distributions in the present study. Using the full conditionals, samples of posterior marginal distributions were generated using the Gibbs sampler and Metropolis-Hastings algorithms and implemented in R. The Bayesian method allowed the use of posterior distributions of another study that evaluated the model used as prior distributions for parameters in the present study. The posterior full conditional distributions for the parameters were normal distributions and gamma distributions, respectively. The Bayesian method was efficient for the study of the first-order kinetic model to describe zinc extraction from soil with sewage sludge using seven extraction solutions.