Gibbs Sampling Research Articles

Bayesian predictive modeling of multi-source multi-way data

A Bayesian approach to predict a continuous or binary outcome from data that are collected from multiple sources with a multi-way (i.e., multidimensional tensor) structure is described. As a motivating example, molecular data from multiple 'omics sources, each measured over multiple developmental time points, as predictors of early-life iron deficiency (ID) in a rhesus monkey model are considered. The method uses a linear model with a low-rank structure on the coefficients to capture multi-way dependence and model the variance of the coefficients separately across each source to infer their relative contributions. Conjugate priors facilitate an efficient Gibbs sampling algorithm for posterior inference, assuming a continuous outcome with normal errors or a binary outcome with a probit link. Simulations demonstrate that the model performs as expected in terms of misclassification rates and correlation of estimated coefficients with true coefficients, with large gains in performance by incorporating multi-way structure and modest gains when accounting for differing signal sizes across the different sources. Moreover, it provides robust classification of ID monkeys for the motivating application.

Bayesian inference for multidimensional graded response model using Pólya-Gamma latent variables

In the context of both cognitive and affective tests, items are usually designed to involve more than two responses, for which polytomous models are applicable. The purpose of this paper is to propose a highly effective Pólya-Gamma Gibbs sampling algorithm based on auxiliary variables to estimate the multidimensional graded response model that has been widely used in psychological, educational, and health-related assessment. The strategy is based on the Pólya Gamma family of distributions which provides a closed-form posterior distribution for logistic-based models. With the introduction of the two latent variables, the full conditional distributions are tractable, and consequently the Gibbs sampling is easy to implement. Nice features including empirical performance of the proposed methodology are demonstrated by simulation studies. Finally, two empirical data sets were analysed to demonstrate the efficiency and utility of the proposed method.

A unique Markov chain Monte Carlo method for forecasting wind power utilizing time series model

Concerns that impair human societies frequently include a heavy dependence on petroleum and coal and emissions of greenhouse gases. Thus, adopting renewable energy sources, such as wind power, has become a practical solution to this problem. Therefore, to carry out the research on wind velocity energy, a time-series structure is necessary. This study uses the Markov chain Monte Carlo approach and the Seasonal Autoregressive Integrated Moving Average (SARIMA) model to estimate short-term and long-term sustained winds. The significance of building a wind energy system is initially discussed, after which a wind velocity time-series framework based on a SARIMA is presented, followed by a short-term and Long-term wind speed projection. Furthermore, a methodology utilizing the Markov chain Monte Carlo method (MCMC) is suggested to establish a wind energy time-series analysis. This framework draws a Markov chain for time-series data on wind energy to maintain stochasticity and realize the probability transition matrix. Gibbs sampling is employed as well. The model's forecasting abilities were tested using the original database and various efficiency assessment measures, including Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE) with efficiency of 13.09 and 1.03. In this study, a framework with the highest KGE and WI as well as the lowest RMSE and MAE was chosen. The findings demonstrate that the approach used in the operation provides outstanding predictability.

Stochastic EM Algorithm for Joint Model of Logistic Regression and Mechanistic Nonlinear Model in Longitudinal Studies

We study a joint model where logistic regression is applied to binary longitudinal data with a mismeasured time-varying covariate that is modeled using a mechanistic nonlinear model. Multiple random effects are necessary to characterize the trajectories of the covariate and the response variable, leading to a high dimensional integral in the likelihood. To account for the computational challenge, we propose a stochastic expectation-maximization (StEM) algorithm with a Gibbs sampler coupled with Metropolis–Hastings sampling for the inference. In contrast with previous developments, this algorithm uses single imputation of the missing data during the Monte Carlo procedure, substantially increasing the computing speed. Through simulation, we assess the algorithm’s convergence and compare the algorithm with more classical approaches for handling measurement errors. We also conduct a real-world data analysis to gain insights into the association between CD4 count and viral load during HIV treatment.

Full Bayesian analysis of double seasonal autoregressive models with real applications

We present a full Bayesian analysis of multiplicative double seasonal autoregressive (DSAR) models in a unified way, considering identification (best subset selection), estimation, and prediction problems. We assume that the DSAR model errors are normally distributed and introduce latent variables for the model lags, and then we embed the DSAR model in a hierarchical Bayes normal mixture structure. By employing the Bernoulli prior for each latent variable and the mixture normal and inverse gamma priors for the DSAR model coefficients and variance, respectively, we derive the full conditional posterior and predictive distributions in closed form. Using these derived conditional posterior and predictive distributions, we present the full Bayesian analysis of DSAR models by proposing the Gibbs sampling algorithm to approximate the posterior and predictive distributions and provide multi-step-ahead predictions. We evaluate the efficiency of the proposed full Bayesian analysis of DSAR models using an extensive simulation study, and we then apply our work to several real-world hourly electricity load time series datasets in 16 European countries.

Efficient Data Augmentation Techniques for Some Classes of State Space Models

Data augmentation improves the convergence of iterative algorithms, such as the EM algorithm and Gibbs sampler by introducing carefully designed latent variables. In this article, we first propose a data augmentation scheme for the first-order autoregression plus noise model, where optimal values of working parameters introduced for recentering and rescaling of the latent states, can be derived analytically by minimizing the fraction of missing information in the EM algorithm. The proposed data augmentation scheme is then utilized to design efficient Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of some non-Gaussian and nonlinear state space models, via a mixture of normals approximation coupled with a block-specific reparametrization strategy. Applications on simulated and benchmark real data sets indicate that the proposed MCMC sampler can yield improvements in simulation efficiency compared with centering, noncentering and even the ancillarity-sufficiency interweaving strategy.

Over-the-Air Antenna Array Calibration for mmWave Hybrid Beamforming Systems Based on Monte Carlo Markov Chain Method

The antenna phase calibration is critical for the millimeter wave (mmWave) hybrid beamforming communication system. Especially, with the increasing number of antennas being deployed in future massive MIMO systems, it is unrealistic to calibrate each antenna path in a microwave anechoic chamber. This work proposes a phase calibration based on over-the-air (OTA) scheme for mmWave hybrid beamforming systems, where the calibration process is performed automatically by the online signal transmission. The challenge for the OTA-based phase calibration is the imperfect channel estimation due to the phase error. To solve the challenge, we simplify the channel estimation process by integrating the beam alignment and phase calibration process. Consequently, the phase calibration is formatted to an MMSE-criterion based optimization problem solved by the Monte Carlo Markov Chain (MCMC) with probabilistic conversion improved Gibbs sampling method. Furthermore, an alternating iteration algorithm is developed to calibrate phase deviation for both the transmitter and receiver. We also extend our method to solve the joint amplitude and phase calibration problem. According to the simulation result, the calibration results outperform existing methods when the signal-to-noise ratio is less than 15dB. Besides, the proposed method exhibits a high beamforming gain during the calibration process, which paves the way for future massive device deployment in a practical system.

Cell-Free UAV Networks: Asymptotic Analysis and Deployment Optimization

Recently, cell-free (CF) architectures, in which every user can potentially communicate with every base station, have received a lot of attention. This paper considers the uplink of fully and partially centralized CF networks where unmanned aerial vehicles serve as flying base stations (FBSs). A subset of FBSs participates in the reception of each user and a subset of users is received by each FBS. Deterministic equivalent expressions, exact asymptotically in the subset sizes and approximate for finite dimensions thereof, are derived for the spectral efficiency under Rician fading. Capitalizing on these expressions, the FBS deployment problem is investigated for different receiver architectures. The nonconvex deployment problem, tackled through a combination of gradient-based and Gibbs sampling algorithms, results in a superior performance with respect to a square grid deployment; this superiority extends to the minimum and aggregate spectral efficiency for both fully and partially centralized cell-free networks.

Surface roughness prediction using multi-source heterogeneous data and Bayesian quantile regression in milling process

To quantify the influence of cutting parameters on surface roughness and improve the prediction accuracy, a novel method is proposed for surface roughness prediction in milling process. A quantile regression (QR) model is constructed using multi-source heterogeneous (MSH) data comprising monitoring signals and cutting parameters within a Bayesian framework. The model parameters are then estimated using Gibbs sampling, and the predicted roughness and confidence interval (CI) can be obtained using the proposed Bayesian quantile model (BQR). Experimental results show that the proposed method has a prediction error of 15.05 %, which is 13.11 %, 0.01 %, and 1.7 % lower than least squares regression (LSR), back propagation network (BPN), and Bayesian linear regression (BLR), respectively. Through the comparison between predictions and experimental measurements, the proposed method has no great advantages over BPN in terms of prediction error, and is even 2.39 % higher than the prediction error of Support Vector Machine (SVM). However, the advantage of the proposed method compared with the above methods is that it can provide the analysis results of the influence of cutting parameters on surface roughness, which has practical significance in providing guidance for operators to reduce surface roughness in the milling process.

Estimates of genetic parameters for indicator traits of resistance to gastrointestinal nematodes and growth traits in Santa Inês sheep

Infections by gastrointestinal nematodes is a major problem in the sheep industry worldwide. Therefore, this study aimed: (1) to estimate genetic parameters for indicator traits of resistance to gastrointestinal nematodes such as packed cell volume (PCV), total plasma protein (TPP), color of the ocular conjunctiva by Famacha© method (FAM), and fecal egg count (FEC); (2) to estimate genetic parameters for growth traits such as body weight (BW), body condition score (BCS) and chest girth (CG); and (3) evaluate the FAM as an indicator trait of resistance to gastrointestinal nematodes in Santa Inês sheep. The dataset contained 3524 observations from 1176 animals and the pedigree presented 2011 animals. The genetic parameters were estimated by the animal model, in single- and multi-trait Bayesian analyses, using the THRGIBBS1F90 program. Stochastic Markov Chain Monte Carlo Simulation (MCMC) was used to obtain estimates of the parameters of interest, using Gibbs sampling. The a posteriori estimates were obtained with the POSTGIBBSF90 program. The sensitivity and specificity of the FAM were 66.7 % and 75.8 %, respectively. In general, the sensitivity and specificity values found herein do not invalidate the use of FAM. In the single-trait model, the heritability estimates for BW, CG, BCS, PCV, TPP, FAM, and FEC were 0.51 ± 0.02, 0.61 ± 0.03, 0.40 ± 0.04, 0.38 ± 0.03, 0.29 ± 0.03, 0.19 ± 0.05 and 0.15 ± 0.03, respectively. In the multi-trait model, the heritability observed for these traits were 0.73 ± 0.02, 0.64 ± 0.02, 0.35 ± 0.05, 0.38 ± 0.03, 0.29 ± 0.03, 0.20 ± 0.05 and 0.13 ± 0.04, respectively. Genetic correlations between growth traits (BW, BCS, and CG) were positive with high magnitude. In addition, growth traits exhibited positive genetic correlations with PCV and TPP, and negative correlations with FAM. The genetic correlation between PCV and FAM, and between FEC and BW were negative and favorable. The genetic parameters suggest that it is possible to obtain genetic gains for the evaluated traits by the response to direct selection. Unfavorable genetic correlations were not found between growth traits and indicator traits of resistance to gastrointestinal nematodes. The heritability of FAM was moderate, and genetic correlations were considered favorable between FAM and PCV. In addition, FAM is easy-to-measure and has a low cost. Therefore, FAM can be an alternative secondary selection criterion associated with body weight (main selection criterion), to increase resistance to gastrointestinal nematodes.

Genetic relationships of body condition score and locomotion with production, type and fertility traits in Holstein-Friesian cows

New traits are sought to add in breeding goals to prevent worsening health and fertility of Polish Holstein-Friesian cows. The objectives of this study were to estimate genetic parameters for body condition score (BCS) and locomotion (LOC) and their relationship with other type traits, milk and fertility traits, and to show genetic trends for BCS and LOC in Polish Holstein-Friesian population. Data on 317 028 Holstein-Friesian cows, born from 2010 through 2015 in 11 792 herds, were collected. All cows were scored for BCS and 43% of them for LOC. All records comprised lactational yields of milk, fat and protein, content of fat and protein and somatic cell count from the first three lactations, stature, five composite and 16 linear conformation traits, and four fertility traits. Genetic parameters were estimated using a Bayesian method with Gibbs Sampling, generating 100 000 samples in each of four steps: BCS and LOC with five composite conformation traits, BCS and LOC with 16 linear conformation traits, BCS and LOC with production traits, and BCS and LOC with four fertility traits. The linear model for BCS and LOC contained fixed effects of herd-year-season-classifier and lactation stage, fixed linear and quadratic regressions on age at calving, fixed linear regression on the percentage of Holstein-Friesian genes, and random additive genetic effect. Breeding values for BCS and LOC were calculated using the same model as used for estimation of genetic parameters. Genetic trends for BCS and LOC, defined as regression coefficients of mean breeding value on birth year, were examined. BCS was a moderately heritable trait (0.19) and was genetically correlated with non-return rate until 56 days after first insemination for cows (−0.32) and with days open (−0.22), so selection for BCS might have a favourable correlated effect on fertility. LOC, lowly heritable (0.06), was relatively strongly genetically correlated with feet-and-legs traits (from 0.48 to 0.93, ignoring sign) and could be included in a selection subindex for feet-and-legs. The positive trend for LOC indicated substantial progress towards the highest genetic value (optimum at the end of the scale), while the small trend for BCS showed a tendency to stabilise the average value in the middle of the scale (optimum for BCS). The estimates of the genetic parameters for BCS and LOC indicate that both traits could contribute to more effective selection to improve fertility (BCS) and legs health (LOC) in the Polish dairy cattle population.

Does climate change affect the transmission of COVID-19? A Bayesian regression analysis.

Coronavirus is an airborne and infectious disease and it is crucial to check the impact of climatic risk factors on the transmission of COVID-19. The main objective of this study is to determine the effect of climate risk factors using Bayesian regression analysis. Coronavirus disease 2019, due to the effect of the SARS-CoV-2 virus, has become a serious global public health issue. This disease was identified in Bangladesh on March 8, 2020, though it was initially identified in Wuhan, China. This disease is rapidly transmitted in Bangladesh due to the high population density and complex health policy setting. To meet our goal, The MCMC with Gibbs sampling is used to draw Bayesian inference, which is implemented in WinBUGS software. The study revealed that high temperatures reduce confirmed cases and deaths from COVID-19, but low temperatures increase confirmed cases and deaths. High temperatures have decreased the proliferation of COVID-19, reducing the virus's survival and transmission. Considering only the existing scientific evidence, warm and wet climates seem to reduce the spread of COVID-19. However, more climate variables could account for explaining most of the variability in infectious disease transmission.

Bayesian Estimation of Fault Slip Distribution for Slow Slip Events Based on an Efficient Hybrid Optimal Directional Gibbs Sampler and Its Application to the Guerrero 2006 Event

We propose an efficient Bayesian approach to infer a fault displacement from geodetic data in a slow slip event. Our physical model of the slip process reduces to a multiple linear regression subject to constraints. Assuming a Gaussian model for the geodetic data and considering a multivariate truncated normal prior distribution for the unknown fault slip, the resulting posterior distribution is also multivariate truncated normal. Regarding the posterior, we propose an algorithm based on Optimal Directional Gibbs that allows us to efficiently sample from the resulting high-dimensional posterior distribution of along dip and along strike movements of our fault grid division. A synthetic fault slip example illustrates the flexibility and accuracy of the proposed approach. The methodology is also applied to a real data set, for the 2006 Guerrero, Mexico, Slow Slip Event, where the objective is to recover the fault slip on a known interface that produces displacements observed at ground geodetic stations. As a by-product of our approach, we are able to estimate moment magnitude for the 2006 Guerrero Event with uncertainty quantification.

Hierarchical reconstruction of 3D well-connected porous media from 2D exemplars using statistics-informed neural network

The relationships between porous microstructures and transport properties are of fundamental importance in various scientific and engineering applications. Due to the intricacy, stochasticity and heterogeneity of porous media, reliable characterization and modeling of transport properties often require a complete dataset of internal microstructure samples. However, it is often an unbearable cost to acquire sufficient 3D digital microstructures by purely using microscopic imaging systems. This paper presents a machine learning-based technique to hierarchically reconstruct 3D well-connected porous microstructures from one isotropic or several anisotropic low-cost 2D exemplar(s). To compactly characterize the large-scale microstructural features, a Gaussian image pyramid is built for each 2D exemplar. Local morphology patterns are collected from the Gaussian image pyramids, and then they serve as the training data to embed the 2D morphological statistics into feed-forward neural networks at multiple length levels. By using a specially-developed morphology integration scheme, the 3D morphological statistics at different levels can be inferred from the statistics-informed neural networks. Gibbs sampling is adopted to hierarchically reconstruct 3D microstructures by using multi-level 3D morphological statistics, where the large-scale, regional and local morphological patterns are statistically generated and successively added to the same 3D random field. The proposed method is tested on a series of porous media with distinct morphologies, and the statistical equivalence between the reconstructed and the real microstructures is systematically evaluated by comparing morphological descriptors and transport properties. The results demonstrate that the proposed 2D-to-3D microstructure reconstruction method is a universal and efficient approach to generating morphologically and physically realistic samples of porous media.

BAYESIAN ANALYSIS OF TOBIT QUANTILE REGRESSION WITH ADAPTIVE LASSO PENALTY IN HOUSEHOLD EXPENDITURE FOR CIGARETTE CONSUMPTION

Tobit Quantile Regression with Adaptive Lasso Penalty is a quantile regression model on censored data that adds Lasso's adaptive penalty to its parameter estimation. The estimation of the regression parameters is solved by Bayesian analysis. Parameters are assumed to follow a certain distribution called the prior distribution. Using the sample information along with the prior distribution, the conditional posterior distribution is searched using the Box-Tiao rule. Computational solutions are solved by the MCMC Gibbs Sampling algorithm. Gibbs Sampling can generate samples based on the conditional posterior distribution of each parameter in order to obtain a posterior joint distribution. Tobit Quantile Regression with Adaptive Lasso Penalty was applied to data on Household Expenditure for Cigarette Consumption in 2011. As a comparison for data analysis, Tobit Quantile Regression was used. The results of data analysis show that the Tobit Quantile Regression model with Adaptive Lasso Penalty is better than the Tobit Quantile Regression.

Probabilistic Models for the Shear Strength of RC Deep Beams

A new shear strength determination of reinforced concrete (RC) deep beams was proposed by using a statistical approach. The Bayesian–MCMC (Markov Chain Monte Carlo) method was introduced to establish a new shear prediction model and to improve seven existing deterministic models with a database of 645 experimental data. The bias correction terms of deterministic models were described by key explanatory terms identified by a systematic removal process. Considering multi-parameters, the Gibbs sampling was used to solve the high dimensional integration problem and to determine optimum and reliable model parameters with 50,000 iterations for probabilistic models. The model continuity and uncertainty for key parameters were quantified by the partial factor that was investigated by comparing test and model results. The partial factor for the proposed model was 1.25. The proposed model showed improved accuracy and continuity with the mean and coefficient of variation (CoV) of the experimental-to-predicted results ratio as 1.0357 and 0.2312, respectively.

A Bayes Analysis of a Dependent Competing Risk Model Based on Marshall–Olkin Bivariate Weibull Distribution

This paper considers a competing risk model defined on the basis of minimum of two dependent failures where the two failures are assumed to jointly follow Marshall–Olkin bivariate Weibull distribution. This paper explores some important features of corresponding likelihood functions and performs a full Bayesian analysis of the model for data resulting from normal as well as accelerated life tests. The accelerated model is described by regressing the scale parameters of the model through inverse power-law relationship. Posterior-based inferences are drawn using the Gibbs sampler algorithm after specifying proper but vague priors for the model parameters. The numerical illustration is provided using real datasets. The performance of the model is assured by Bayesian tools of model compatibility and then the entertained model is compared with the competing risk model based on Marshall–Olkin bivariate exponential assumption.

Functional principal component models for sparse and irregularly spaced data by Bayesian inference

The area of functional principal component analysis (FPCA) has seen relatively few contributions from the Bayesian inference. A Bayesian method in FPCA is developed under the cases of continuous and binary observations for sparse and irregularly spaced data. In the proposed Markov chain Monte Carlo (MCMC) method, Gibbs sampler approach is adopted to update the different variables based on their conditional posterior distributions. In FPCA, a set of eigenfunctions is suggested under Stiefel manifold, and samples are drawn from a Langevin–Bingham matrix variate distribution. Penalized splines are used to model mean trajectory and eigenfunction trajectories in generalized functional mixed models; and the proposed model is casted into a mixed-effects model framework for Bayesian inference. To determine the number of principal components, reversible jump Markov chain Monte Carlo (RJ-MCMC) algorithm is implemented. Four different simulation settings are conducted to demonstrate competitive performance against non-Bayesian approaches in FPCA. Finally, the proposed method is illustrated to the analysis of body mass index (BMI) data by gender and ethnicity.

A Bayesian model for multivariate discrete data using spatial and expert information with application to inferring building attributes

When modeling sparsely observed multivariate data, strong prior information elicited from experts can be used to bolster predictive accuracy and counteract sampling bias. Similarly, modeling autocorrelation in space can help make use of co-occurrence patterns present in many types of spatial data. To make use of both expert prior information and spatial structure, we propose a novel graphical model for a spatial Bayesian network developed specifically to address challenges in inferring the attributes of buildings from geographically sparse observational data. This model is implemented as the sum of a spatial multivariate Gaussian random field and a tabular conditional probability function in real-valued space prior to projection onto the probability simplex. This modeling form is especially suitable for the usage of prior information in the form of sets of atomic rules obtained from experts. To perform inference with missing data, we implement a Markov chain Monte Carlo scheme composed of alternating steps of Gibbs sampling of missing entries and Hamiltonian Monte Carlo for model parameters. A case study in building attribution is presented to highlight the advantages and limitations of this approach.

The Process Capability Index of Pareto Model under Progressive Type-II Censoring: Various Bayesian and Bootstrap Algorithms for Asymmetric Data

It is agreed by industry experts that manufacturing processes are evaluated using quantitative indicators of units produced from this process. For example, the Cpy process capability index is usually unknown and therefore estimated based on a sample drawn from the requested process. In this paper, Cpy process capability index estimates were generated using two iterative methods and a Bayesian method of estimation based on stepwise controlled type II data from the Pareto model. In iterative methods, besides the traditional probability-based estimation, there are other competitive methods, known as bootstrap, which are alternative methods to the common probability method, especially in small samples. In the Bayesian method, we have applied the Gibbs sampling procedure with the help of the significant sampling technique. Moreover, the approximate and highest confidence intervals for the posterior intensity of Cpy were also obtained. Massive simulation studies have been performed to evaluate the behavior of Cpy. Ultimately, application to real-life data is seen to demonstrate the proposed methodology and its applicability.