Full Conditional Distributions Research Articles

Constrained estimation for the binomial AR(1) model: on Bayesian approach

In this paper, we consider the constrained estimation problem for the binomial AR(1) model using Bayesian approach. We show that by using Gibbs sampling algorithm, the constrains for the parameters and latent variables can be routinely implemented. While obtaining all the full conditional distributions under the unconstrained environment, we only need to generate samples from them and make restrictions to easily described cross-sections, respectively, which avoids complex numerical integrations under the overall constraint set. In the simulation study, the performance of our algorithm is checked. Finally, the method is applied to two real data examples.

Group Inverse-Gamma Gamma Shrinkage for Sparse Linear Models with Block-Correlated Regressors

Heavy-tailed continuous shrinkage priors, such as the horseshoe prior, are widely used for sparse estimation problems. However, there is limited work extending these priors to predictors with grouping structures. Of particular interest in this article, is regression coefficient estimation where pockets of high collinearity in the covariate space are contained within known covariate groupings. To assuage variance inflation due to multicollinearity we propose the group inverse-gamma gamma (GIGG) prior, a heavy-tailed prior that can trade-off between local and group shrinkage in a data adaptive fashion. A special case of the GIGG prior is the group horseshoe prior, whose shrinkage profile is correlated within-group such that the regression coefficients marginally have exact horseshoe regularization. We show posterior consistency for regression coefficients in linear regression models and posterior concentration results for mean parameters in sparse normal means models. The full conditional distributions corresponding to GIGG regression can be derived in closed form, leading to straightforward posterior computation. We show that GIGG regression results in low mean-squared error across a wide range of correlation structures and within-group signal densities via simulation. We apply GIGG regression to data from the National Health and Nutrition Examination Survey for associating environmental exposures with liver functionality.

Robust Bayesian nonparametric variable selection for linear regression

AbstractSpike‐and‐slab and horseshoe regressions are arguably the most popular Bayesian variable selection approaches for linear regression models. However, their performance can deteriorate if outliers and heteroskedasticity are present in the data, which are common features in many real‐world statistics and machine learning applications. This work proposes a Bayesian nonparametric approach to linear regression that performs variable selection while accounting for outliers and heteroskedasticity. Our proposed model is an instance of a Dirichlet process scale mixture model with the advantage that we can derive the full conditional distributions of all parameters in closed‐form, hence producing an efficient Gibbs sampler for posterior inference. Moreover, we present how to extend the model to account for heavy‐tailed response variables. The model's performance is tested against competing algorithms on synthetic and real‐world datasets.

Approximate inferences for Bayesian hierarchical generalised linear regression models

SummaryGeneralised linear mixed regression models are fundamental in statistics. Modelling random effects that are shared by individuals allows for correlation among those individuals. There are many methods and statistical packages available for analysing data using these models. Most require some form of numerical or analytic approximation because the likelihood function generally involves intractable integrals over the latents. The Bayesian approach avoids this issue by iteratively sampling the full conditional distributions for various blocks of parameters and latent random effects. Depending on the choice of the prior, some full conditionals are recognisable while others are not. In this paper we develop a novel normal approximation for the random effects full conditional, establish its asymptotic correctness and evaluate how well it performs. We make the case for hierarchical binomial and Poisson regression models with canonical link functions, for hierarchical gamma regression models with log link and for other cases. We also develop what we term a sufficient reduction (SR) approach to the Markov Chain Monte Carlo algorithm that allows for making inferences about all model parameters by replacing the full conditional for the latent variables with a considerably reduced dimensional function of the latents. We expect that this approximation could be quite useful in situations where there are a very large number of latent effects, which may be occurring in an increasingly ‘Big Data’ world. In the sequel, we compare our methods with INLA, which is a particularly popular method and which has been shown to be excellent in terms of speed and accuracy across a variety of settings. Our methods appear to be comparable to theirs in terms of accuracy, while INLA was faster, for the settings we considered. In addition, we note that our methods and those of others that involve Gibbs sampling trivially handle parameters that are functions of multiple parameters, while INLA approximations do not. Our primary illustration is for a three‐level hierarchical binomial regression model for data on health outcomes for patients who are clustered within physicians who are clustered within particular hospitals or hospital systems.

Gibbs sampler for Bayesian prediction of triple seasonal autoregressive processes

Researchers have extended autoregressive (AR) time-series models to adequately fit and model time-series with triple seasonality. These AR extensions can be referred to as triple seasonal AR (TSAR) models. For these TSAR time-series models, only Bayesian estimation and identification have been introduced. Therefore, in this article, we aim to extend the existing work for presenting the Bayesian prediction for TSAR models using the Gibbs sampler algorithm. In this Bayesian prediction, we first assume the TSAR errors are identically normally distributed, and we employ normal-gamma and g priors for the TSAR parameters. Based on the normally distributed TSAR errors and the specified TSAR parameters’ priors, we are able to derive the conditional predictive and posterior densities. Particularly, we show the conditional posterior densities of TSAR coefficients and variance are the multivariate normal and inverse-gamma, respectively. In addition, for the future TSAR observations, we show the conditional predictive density is the multivariate normal. Using these derived full conditional distributions, we propose the Gibbs sampler to efficiently approximate the joint predictive and posterior densities and to easily carry out multiple-step ahead predictions. We validate the accuracy of forecasting for the proposed Gibbs sampler by conducting a simulation study. Moreover, we apply the proposed Gibbs sampler to hourly electricity-load time-series datasets in some European countries, along with a comparison with the well-known long-short term memory (LSTM) recurrent neural model.

Covariate-Dependent Clustering of Undirected Networks with Brain-Imaging Data

This article focuses on model-based clustering of subjects based on the shared relationships of subject-specific networks and covariates in scenarios when there are differences in the relationship between networks and covariates for different groups of subjects. It is also of interest to identify the network nodes significantly associated with each covariate in each cluster of subjects. To address these methodological questions, we propose a novel nonparametric Bayesian mixture modeling framework with an undirected network response and scalar predictors. The symmetric matrix coefficients corresponding to the scalar predictors of interest in each mixture component involve low-rankness and group sparsity within the low-rank structure. While the low-rank structure in the network coefficients adds parsimony and computational efficiency, the group sparsity within the low-rank structure enables drawing inference on network nodes and cells significantly associated with each scalar predictor. Being a principled Bayesian mixture modeling framework, our approach allows model-based identification of the number of clusters, offers clustering uncertainty in terms of the co-clustering matrix and presents precise characterization of uncertainty in identifying network nodes significantly related to a predictor in each cluster. Empirical results in various simulation scenarios illustrate substantial inferential gains of the proposed framework in comparison with competitors. Analysis of a real brain connectome dataset using the proposed method provides interesting insights into the brain regions of interest (ROIs) significantly related to creative achievement in each cluster of subjects. Supplementary material shows the convergence rate for the posterior predictive density of the proposed model, additional simulation examples with model misspecification, full conditional distributions to run the Markov chain Monte Carlo (MCMC) algorithm and also presents traceplots for various model parameters to demonstrate convergence of the MCMC algorithm.

Bayesian Regression Analysis using Median Rank Set Sampling

Bayesian estimation of the linear regression parameter system is considered by deploying Median Rank Set Sampling (MRSS). The full conditional distributions and the associated posterior distribution are obtained. Therefore, based on Markov Chain Monte Carlo simulation, the Bayesian point estimates and credible intervals for the regression parameters are determined. To measure the efficiency of the obtained Bayesian estimates concerning the frequentist estimates we compute the asymptotic relative efficiency of the obtained Bayesian estimates using Markov Chain Monte Carlo simulation.

Fin-GAN: forecasting and classifying financial time series via generative adversarial networks

We investigate the use of Generative Adversarial Networks (GANs) for probabilistic forecasting of financial time series. To this end, we introduce a novel economics-driven loss function for the generator. This newly designed loss function renders GANs more suitable for a classification task, and places them into a supervised learning setting, whilst producing full conditional probability distributions of price returns given previous historical values. Our approach moves beyond the point estimates traditionally employed in the forecasting literature, and allows for uncertainty estimates. Numerical experiments on equity data showcase the effectiveness of our proposed methodology, which achieves higher Sharpe Ratios compared to classical supervised learning models, such as LSTMs and ARIMA.

Nonparametric Bayesian modeling for non-normal data through a transformation

<abstract><p>In many applications, modeling based on a normal kernel is preferred because not only does the normal kernel belong to the family of stable distributions, but also it is easy to satisfy the stationary condition in the stochastic process. However, the characteristic of the data, such as count or proportion, is a major obstacle to complete modeling based on a normal distribution. To solve a limited boundary or non-normal distribution problem, we provided a novel transformation method and proposed a nonparametric Bayesian approach based on a normal kernel of the transformed variable. In particular, the provided transformation transforms any probability space into a real space and is free from the constraints of the previous transformation, such as skewness, presence of power, and bounded domains. Another advantage was that it was possible to use the Dirichlet process mixture model with full conditional posterior distributions for all parameters, leading to a fast convergence rate in the Markov chain Monte Carlo. The proposed methodology was illustrated with simulated datasets and two real datasets with non-normal distribution problems. In addition, to demonstrate the superiority of the proposed methodology, the comparison with the transformed Bernstein polynomial model was made in the real data analysis.</p></abstract>

Bayesian boundary trend filtering

Estimating boundary curves has many applications such as economics, climate science, and medicine. Bayesian trend filtering has been developed as one of locally adaptive smoothing methods to estimate the non-stationary trend of data. This paper develops a Bayesian trend filtering for estimating the boundary trend. To this end, the truncated multivariate normal working likelihood and global-local shrinkage priors based on the scale mixtures of normal distribution are introduced. In particular, well-known horseshoe prior for difference leads to locally adaptive shrinkage estimation for boundary trend. However, the full conditional distributions of the Gibbs sampler involve high-dimensional truncated multivariate normal distribution. To overcome the difficulty of sampling, an approximation of truncated multivariate normal distribution is employed. Using the approximation, the proposed models lead to an efficient Gibbs sampling algorithm via the Pólya-Gamma data augmentation. The proposed method is also extended by considering a nearly isotonic constraint. The performance of the proposed method is illustrated through some numerical experiments and real data examples.

Computationally Efficient Poisson Time-Varying Autoregressive Models through Bayesian Lattice Filters

Estimation of time-varying autoregressive models for count-valued time series can be computationally challenging. In this direction, we propose a time-varying Poisson autoregressive (TV-Pois-AR) model that accounts for the changing intensity of the Poisson process. Our approach can capture the latent dynamics of the time series and therefore make superior forecasts. To speed up the estimation of the TV-AR process, our approach uses the Bayesian Lattice Filter. In addition, the No-U-Turn Sampler (NUTS) is used, instead of a random walk Metropolis–Hastings algorithm, to sample intensity-related parameters without a closed-form full conditional distribution. The effectiveness of our approach is evaluated through model-based and empirical simulation studies. Finally, we demonstrate the utility of the proposed model through an example of COVID-19 spread in New York State and an example of US COVID-19 hospitalization data.

Structured Shrinkage Priors

In many regression settings the unknown coefficients may have some known structure, for instance they may be ordered in space or correspond to a vectorized matrix or tensor. At the same time, the unknown coefficients may be sparse, with many nearly or exactly equal to zero. However, many commonly used priors and corresponding penalties for coefficients do not encourage simultaneously structured and sparse estimates. In this article we develop structured shrinkage priors that generalize multivariate normal, Laplace, exponential power and normal-gamma priors. These priors allow the regression coefficients to be correlated a priori without sacrificing elementwise sparsity or shrinkage. The primary challenges in working with these structured shrinkage priors are computational, as the corresponding penalties are intractable integrals and the full conditional distributions that are needed to approximate the posterior mode or simulate from the posterior distribution may be nonstandard. We overcome these issues using a flexible elliptical slice sampling procedure, and demonstrate that these priors can be used to introduce structure while preserving sparsity. Supplementary materials for this article are available online.

The Influence of Equity Market Sentiment on Credit Default Swap Markets: Evidence from Wavelet Quantile Regression

Previous studies focused on the fundamental channels of the interaction between the equity market and credit default swap (CDS) market. This paper finds another channel, investor sentiment, that contributes to the impact of the equity market on the CDS market under different time horizons and market conditions within the framework of wavelet quantile regression. It absorbs both the merits of wavelet transform and quantile regression and is advantageous in analyzing heterogeneous time horizons and full conditional distributions. Empirical results show that investor attitude turning optimistic has a negative influence on the deviation of CDS market spread from theoretical value, while the intensification of fear among equity market will enlarge this deviation. Besides, we discovered that the influence of equity market sentiment on the CDS market first increases and then decreases as the time horizon lengthens and that the greater the deviation of CDS spreads from intrinsic value is, the more irrational the CDS market participants are. These findings suggest that the influence of investor sentiment on the credit default swap market is self-reinforced. Our results are robust after controlling for macroeconomic conditions and under different wavelet decompositions. Reasonable suggestions are given to financial institutions, investors, and policy makers based on our findings.

Virtual sensing based on Hierarchical Bayesian Modeling framework using a Laplace-based Gibbs sampler

This paper presents a Bayesian-based approach to realize virtual strain sensing and quantify the uncertainty in these estimates using sparse output-only measurements. This method relies on output-only measurements to identify the modal coordinates of the structure and realizes the virtual sensing according to the modal superposition method. A robust Hierarchical Bayesian modeling (HBM) framework is developed to fully account for the uncertainties arising from modeling errors and measurement noise. Moreover, the HBM framework has the ability to quantify the uncertainty of parameters in the statistical model by introducing an additional layer of the prior distribution. The Gibbs sampler is adopted to obtain the marginal posterior distributions of the hyper-parameters, with the implementation of Gibbs methods relying on the derived analytical expressions of the full conditional posterior distributions. The Laplace asymptotic approximation simplified the form of the full conditional posterior distributions, thus improving the computational efficiency. Given the hyper-parameters’ marginal posterior distribution, the dynamic parameters’ marginal posterior distribution can be calculated according to the total probability theorem. Finally, one can propagate the uncertainty to make predictions of quantities of interest (QoI). Three case studies, including two numerical examples and a laboratory model test, are carried out to verify the accuracy and efficiency of the proposed algorithm. The results show that the estimates of the parameters are close to the true values. Besides, the prediction results of QoI also show high accuracy, verifying the effectiveness of the proposed HBM method in virtual sensing.

On MCMC sampling in random coefficients self-exciting integer-valued threshold autoregressive processes

In this study, Bayesian estimation is performed for a class of random coefficient self-exciting integer-valued threshold autoregressive processes with explanatory variables. A new model with a linear structure is obtained through model reconstruction, which makes Markov Chain Monte Carlo method easy to perform. By introducing the latent variables series, a complete data likelihood is obtained. Based on this likelihood, the full conditional distributions are easily obtained for all the parameters and latent variables. By maximizing the posterior probability function, the threshold parameter is accurately estimated. Finally, some numerical results of the estimates and a real data example of crime counts in Ballina, New South Wales, Australia are presented.

Validation of uncertainty predictions in digital soil mapping

It is quite common in digital soil mapping (DSM) to quantify the uncertainty of issued predictions, that is to make probabilistic predictions. Yet, little attention has been paid to its validation. Probabilistic predictions are only of value for end users if they are reliable and ideally also sharp. Reliability refers to the consistency between predicted conditional probabilities and observed frequencies of independent test data. Sharpness refers to the concentration of a conditional probability distribution function, i.e. its narrowness. The prediction interval coverage probability (PICP) is currently used in DSM to validate the reliability of prediction intervals but it is ignorant of a potential one-sided bias of its boundaries. Therefore, we propose to extend the current validation procedure with metrics used in the broader probabilistic literature. These metrics not only evaluate probabilistic predictions in prediction interval format but also quantiles or full conditional probability distributions. We suggest the quantile coverage probability (QCP) and probability integral transform (PIT) histogram as alternatives to PICP and proper scoring rules for relative comparisons of competing probabilistic models. As scoring rules, we present the interval score (IS) and the continuous ranked probability score (CRPS), which can be decomposed into a reliability part (RELI). We illustrated the use of these metrics in a case study using soil pH and soil organic carbon from the LUCAS-soil database. Thereby, probabilistic predictions of five different models were compared: a reference null model (NM), quantile regression forest (QRF), quantile regression post-processing of a random forest (QRPP RF), kriging with external drift (KED) and quantile regression neural network (QRNN). For KED and QRNN, one-sided bias was found. This was not apparent from PICP but was shown by use of the PIT histogram and QCP. RELI summarized the trends found in QCP, PICP and PIT histograms to one numerical value. CRPS and IS were especially harsh to outliers and low sharpness. According to CRPS and IS, the best probabilistic predictions were obtained by QRF and QRPP RF and the worst by NM.

Bayesian Subset Selection of Seasonal Autoregressive Models

Seasonal autoregressive (SAR) models have many applications in different fields, such as economics and finance. It is well known in the literature that these models are nonlinear in their coefficients and that their Bayesian analysis is complicated. Accordingly, choosing the best subset of these models is a challenging task. Therefore, in this paper, we tackled this problem by introducing a Bayesian method for selecting the most promising subset of the SAR models. In particular, we introduced latent variables for the SAR model lags, assumed model errors to be normally distributed, and adopted and modified the stochastic search variable selection (SSVS) procedure for the SAR models. Thus, we derived full conditional posterior distributions of the SAR model parameters in the closed form, and we then introduced the Gibbs sampler, along with SSVS, to present an efficient algorithm for the Bayesian subset selection of the SAR models. In this work, we employed mixture–normal, inverse gamma, and Bernoulli priors for the SAR model coefficients, variance, and latent variables, respectively. Moreover, we introduced a simulation study and a real-world application to evaluate the accuracy of the proposed algorithm.

Bayesian inference for multidimensional graded response model using Pólya-Gamma latent variables

In the context of both cognitive and affective tests, items are usually designed to involve more than two responses, for which polytomous models are applicable. The purpose of this paper is to propose a highly effective Pólya-Gamma Gibbs sampling algorithm based on auxiliary variables to estimate the multidimensional graded response model that has been widely used in psychological, educational, and health-related assessment. The strategy is based on the Pólya Gamma family of distributions which provides a closed-form posterior distribution for logistic-based models. With the introduction of the two latent variables, the full conditional distributions are tractable, and consequently the Gibbs sampling is easy to implement. Nice features including empirical performance of the proposed methodology are demonstrated by simulation studies. Finally, two empirical data sets were analysed to demonstrate the efficiency and utility of the proposed method.

Full Bayesian analysis of double seasonal autoregressive models with real applications

We present a full Bayesian analysis of multiplicative double seasonal autoregressive (DSAR) models in a unified way, considering identification (best subset selection), estimation, and prediction problems. We assume that the DSAR model errors are normally distributed and introduce latent variables for the model lags, and then we embed the DSAR model in a hierarchical Bayes normal mixture structure. By employing the Bernoulli prior for each latent variable and the mixture normal and inverse gamma priors for the DSAR model coefficients and variance, respectively, we derive the full conditional posterior and predictive distributions in closed form. Using these derived conditional posterior and predictive distributions, we present the full Bayesian analysis of DSAR models by proposing the Gibbs sampling algorithm to approximate the posterior and predictive distributions and provide multi-step-ahead predictions. We evaluate the efficiency of the proposed full Bayesian analysis of DSAR models using an extensive simulation study, and we then apply our work to several real-world hourly electricity load time series datasets in 16 European countries.

Bayesian analysis of seasonally cointegrated VAR models

The aim is to develop a Bayesian seasonally cointegrated model for quarterly data. Relevant prior structure is proposed, and the set of full conditional posterior distributions is derived, enabling us to employ the Gibbs sampler for posterior inference. The identification of cointegrating spaces is obtained by orthonormality restrictions imposed on vectors spanning them. The point estimation of the cointegrating spaces is also discussed. In the presence of a seasonal pattern with one cycle per year, the cointegrating vectors belong to the complex space, which should be taken into account in the identification scheme. The methodology is illustrated by the analysis of money and prices in the Polish economy.