Bayesian Inference Procedure Research Articles

Incorporating Different Sources of Information for Bayesian Optimal Portfolio Selection

This article introduces Bayesian inference procedures for tangency portfolios, with a primary focus on deriving a new conjugate prior for portfolio weights. This approach not only enables direct inference about the weights but also seamlessly integrates additional information into the prior specification. Specifically, it automatically incorporates high-frequency returns and a market condition metric (MCM), exemplified by the CBOE Volatility Index (VIX) and Economic Policy Uncertainty Index (EPU), significantly enhancing the decision-making process for optimal portfolio construction. While the Jeffreys’ prior is also acknowledged, emphasis is placed on the advantages and practical applications of the conjugate prior. An extensive empirical study reveals that our method, leveraging this conjugate prior, consistently outperforms existing trading strategies in the majority of examined cases.

Objective Bayesian Meta-Analysis Based on Generalized Marginal Multivariate Random Effects Model

Objective Bayesian inference procedures are derived for the parameters of the multivariate random effects model generalized to elliptically contoured distributions. The posterior for the overall mean vector and the between-study covariance matrix is deduced by assigning two noninformative priors to the model parameter, namely the Berger and Bernardo reference prior and the Jeffreys prior, whose analytical expressions are obtained under weak distributional assumptions. It is shown that the only condition needed for the posterior to be proper is that the sample size is larger than the dimension of the data-generating model, independently of the class of elliptically contoured distributions used in the definition of the generalized multivariate random effects model. The theoretical findings of the paper are applied to real data consisting of ten studies about the effectiveness of hypertension treatment for reducing blood pressure where the treatment effects on both the systolic blood pressure and diastolic blood pressure are investigated.

Enabling identification of component processes in perceptual learning with nonparametric hierarchical Bayesian modeling.

Perceptual learning is a multifaceted process, encompassing general learning, between-session forgetting or consolidation, and within-session fast relearning and deterioration. The learning curve constructed from threshold estimates in blocks or sessions, based on tens or hundreds of trials, may obscure component processes; high temporal resolution is necessary. We developed two nonparametric inference procedures: a Bayesian inference procedure (BIP) to estimate the posterior distribution of contrast threshold in each learning block for each learner independently and a hierarchical Bayesian model (HBM) that computes the joint posterior distribution of contrast threshold across all learning blocks at the population, subject, and test levels via the covariance of contrast thresholds across blocks. We applied the procedures to the data from two studies that investigated the interaction between feedback and training accuracy in Gabor orientation identification over 1920 trials across six sessions and estimated learning curve with block sizes L = 10, 20, 40, 80, 160, and 320 trials. The HBM generated significantly better fits to the data, smaller standard deviations, and more precise estimates, compared to the BIP across all block sizes. In addition, the HBM generated unbiased estimates, whereas the BIP only generated unbiased estimates with large block sizes but exhibited increased bias with small block sizes. With L = 10, 20, and 40, we were able to consistently identify general learning, between-session forgetting, and rapid relearning and adaptation within sessions. The nonparametric HBM provides a general framework for fine-grained assessment of the learning curve and enables identification of component processes in perceptual learning.

Gibbs sampler approach for objective Bayesian inference in elliptical multivariate meta-analysis random effects model

Bayesian inference procedures for the parameters of the multivariate random effects model are derived under the assumption of an elliptically contoured distribution when the Berger and Bernardo reference and the Jeffreys priors are assigned to the model parameters. A new numerical algorithm for drawing samples from the posterior distribution is developed, which is based on the hybrid Gibbs sampler. The new approach is compared to the two Metropolis-Hastings algorithms previously derived in the literature via an extensive simulation study. The findings are applied to a Bayesian multivariate meta-analysis, conducted using the results of ten studies on the effectiveness of a treatment for hypertension. The analysis investigates the treatment effects on systolic and diastolic blood pressure. The second empirical illustration deals with measurement data from the CCAUV.V-K1 key comparison, aiming to compare measurement results of sinusoidal linear accelerometers at four frequencies.

Influence of modeling choices and prior information on the Bayesian assessment of a reinforced concrete bridge

AbstractA lot of bridges are aging and reaching the anticipated service life. To gain insight in their remaining resistance, inspections and measurements can be performed. The resulting information and data can be used to update variables in the degradation models of these bridges. To account for spatial variation, degradation variables can be modeled with random fields. The random fields of the degradation variables are updated based on a Bayesian inference procedure, where different types of heterogenous and indirect data are accounted for. Nevertheless, in this procedure, various assumptions need to be made, such as the quantification of measurement uncertainty, the structural and degradation model to be used, the prior distributions of the variables of interest, and so forth. These assumptions can influence the results of the Bayesian inference. Hence, in this work, it will be investigated what the influence is of different assumptions and how they affect the localization and quantification of damage. This will be done by application to a reinforced concrete girder bridge. The main conclusions are that the model used in the Bayesian inference procedure should resemble the actual structure as good as possible and that simplifications that are allowed in the design can lead to posterior distributions deviating a lot from the actual situation. Moreover, it is illustrated how information from visual observations can be included in the definition of the prior distributions and how this has a beneficial effect on the posterior predictions.

Bayesian Inference for High Dimensional Cox Models with Gaussian and Diffused-Gamma Priors: A Case Study of Mortality in COVID-19 Patients Admitted to the ICU.

Bayesian approaches have been utilized to address the challenge of variable selection and statistical inference in high-dimensional survival analysis. However, the discontinuity of the -norm prior, including the useful spike-and-slab prior, may lead to computational and implementation challenges, potentially limiting the widespread use of Bayesian methods. The Gaussian and diffused-gamma (GD) prior has emerged as a promising alternative due to its continuous-and-differentiable -norm approximation and computational efficiency in generalized linear models. In this paper, we extend the GD prior to semi-parametric Cox models by proposing a rank-based Bayesian inference procedure with the Cox partial likelihood. We develop a computationally efficient algorithm based on the iterative conditional mode (ICM) and Markov chain Monte Carlo methods for posterior inference. Our simulations demonstrate the effectiveness of the proposed method, and we apply it to an electronic health record dataset to identify risk factors associated with COVID-19 mortality in ICU patients at a regional medical center.

Bayesian estimation in multivariate inter-laboratory studies with unknown covariance matrices

In the paper we present Bayesian inference procedures for the parameters of multivariate random effects model, which is used as a quantitative tool for performing multivariate key comparisons and multivariate inter-laboratory studies. The developed new approach does not require that the reported covariance matrices of participating laboratories are known and, as such, it can be used when they are estimated from the measurement results. The Bayesian inference procedures are based on samples generated from the derived posterior distribution when the Berger and Bernardo reference prior and the Jeffreys prior are assigned to the model parameter. Three numerical algorithms for the construction of Markov chains are provided and implemented in the CCAUV.V-K1 key comparisons. All three approaches yield similar Bayesian estimators with wider credible intervals when the Berger and Bernardo reference prior is used. Also, the Bayesian estimators for the elements of the inter-laboratory covariance matrix are larger under this prior than for the Jeffreys prior. Finally, the constructed joint credible sets for the components of the overall mean vector indicate the presence of linear dependence between them which cannot be captured when only univariate key comparisons are performed.

An efficient off-grid direction-of-arrival estimation method based on inverse-free sparse Bayesian learning

The direction-of-arrival (DOA) estimation problem can be formulated in a Bayesian framework and solved well through sparse Bayesian learning (SBL). However, SBL-based methods often suffer from high computational complexity due to the matrix inversion of a large matrix in the Bayesian inference procedure. To address this issue, we propose an efficient DOA estimation method based on inverse-free sparse Bayesian learning (IFSBL). A real-valued vectorized array covariance model and a hierarchical Bayesian model are established. Then, a relaxed lower bound on the likelihood, namely relaxed evidence lower bound (relaxed-ELBO), is obtained as the optimization objective of the Bayesian inference. By utilizing the variational expectation–maximization method for the Bayesian inference, the estimation of the statistics of interest can be achieved, which is free of matrix inversion as the matrix to be inverted is diagonal. Moreover, when we take the off-grid DOAs into account, an efficient grid refinement is developed to iteratively modify the DOA estimates derived from the estimated spatial spectrum. The off-grid error obtained by the root of a simple polynomial is used to compensate for the corresponding DOA estimate. Simulations show good accuracy and improved computational efficiency of the proposed method compared with other state-of-art methods. Further experimental results verify the advantage in the runtime for the proposed method.

Randomization-based, Bayesian inference of causal effects

Abstract Bayesian causal inference in randomized experiments usually imposes model-based structure on potential outcomes. Yet causal inferences from randomized experiments are especially credible because they depend on a known assignment process, not a probability model of potential outcomes. In this article, I derive a randomization-based procedure for Bayesian inference of causal effects in a finite population setting. I formally show that this procedure satisfies Bayesian analogues of unbiasedness and consistency under weak conditions on a prior distribution. Unlike existing model-based methods of Bayesian causal inference, my procedure supposes neither probability models that generate potential outcomes nor independent and identically distributed random sampling. Unlike existing randomization-based methods of Bayesian causal inference, my procedure does not suppose that potential outcomes are discrete and bounded. Consequently, researchers can reap the benefits of Bayesian inference without sacrificing the properties that make inferences from randomized experiments especially credible in the first place.

Reliability-based structural assessment of historical masonry arch bridges: The case study of Cernadela bridge

Nowadays, several historical masonry arch bridges present a deficient state of conservation due to degradation processes induced by natural or human actions. Usually, these constructions have significant economic, cultural, and heritage value. Therefore, they shall be thoroughly assessed to verify their structural integrity and safety condition. For this purpose, reliability-based structural assessments are typically performed. However, the associated outcomes (i.e., reliability index and probability of failure) highly rely on the accuracy of the structural parameters uncertainty quantification. This work presents a study regarding the influence of the scattering of the arches' thickness dimensions in the load-carrying capacity assessment of the Cernadela Bridge, a historical stone bridge located in Galicia, Spain. The study first involved a comprehensive experimental campaign to characterize the outer and inner bridge geometry through geomatic techniques, i.e., terrestrial laser scanning and ground penetrating radar. Subsequently, a limit analysis model was developed, considering only the arches' outer (visible) data. From this initial structural assessment, a reliability index of 2.38 was obtained. The influence of the uncertain structural parameters, both geometric features and material properties, in the collapse load was investigated through a global variance-based sensitivity analysis (i.e., Sobol' indices) complemented by a surrogate modeling strategy based on the Kriging approach. Finally, the measured inner geometry of the arches was introduced in the computational model through Bayesian inference procedures. Thus, two new structural assessments were performed: first, by considering the updated distributions of all arches thicknesses, and second, by considering only the updated distributions of the non-influential ones. Reliability indexes of 1.51 and 2.33 were derived, thus highlighting the importance of a proper uncertainty quantification process and the relevance of the sensitivity analysis outcomes to identify the critical parameters on the bridge mechanical response.

Hierarchical Bayesian perceptual template modeling of mechanisms of spatial attention in central and peripheral cuing.

The external noise paradigm and perceptual template model (PTM) have successfully been applied to characterize observer properties and mechanisms of observer state changes (e.g. attention and perceptual learning) in several research domains, focusing on individual level analysis. In this study, we developed a new hierarchical Bayesian perceptual template model (HBPTM) to model the trial-by-trial data from all individuals and conditions in a published spatial cuing study within a single structure and compared its performance to that of a Bayesian Inference Procedure (BIP), which separately infers the posterior distributions of the model parameters for each individual subject without the hierarchical structure. The HBPTM allowed us to compute the joint posterior distribution of the hyperparameters and parameters at the population, observer, and experiment levels and make statistical inferences at all these levels. In addition, we ran a large simulation study that varied the number of observers and number of trials in each condition and demonstrated the advantage of the HBPTM over the BIP across all the simulated datasets. Although it is developed in the context of spatial attention, the HBPTM and its extensions can be used to model data from the external noise paradigm in other domains and enable predictions of human performance at both the population and individual levels.

Gestational exposure to organophosphate esters and infant anthropometric measures in the first 4 weeks after birth.

Few studies have examined whether gestational exposure to organophosphate esters (OPEs), widely used chemicals with potential endocrine-disrupting potency and developmental toxicity, is associated with impaired infant growth. We analyzed data from 329 mother-infant pairs in the Health Outcomes and Measures of the Environment (HOME) Study (2003-2006, Cincinnati, Ohio, USA). We quantified concentrations of four OPE metabolites in maternal urine collected at 16 and 26 weeks of gestation, and at delivery. We calculated z-scores using 2006 World Health Organization (WHO) child growth standards for the 4-week anthropometric measures (weight, length, and head circumference), the ponderal index, and weekly growth rates. We used multiple informant models to examine window-specific associations between individual OPE metabolites and anthropometric outcomes. We further modeled OPEs as a mixture for window-specific associations with 4-week anthropometric outcomes using mean field variational Bayesian inference procedure for lagged kernel machine regression (MFVB-LKMR). We stratified the models by infant sex. Diphenyl phosphate (DPHP) in mothers at 16 weeks, and bis(2-chloroethyl) phosphate (BCEP) and bis(1,3-dichloro-2-propyl) phosphate (BDCIPP) at delivery were positively associated with z-scores of weight, length, and head circumference in all infants at 4 weeks of age. After stratifying by infant sex, positive associations were only observed in males for DPHP at 16 weeks and BCEP at delivery and in females for BDCIPP at delivery. Negative associations not present in all infants were observed in males for di-n-butyl phosphate (DNBP) at 26 weeks of gestation with weight z-score and DPHP at delivery with head circumference z-score. Results were generally similar using MFVB-LKMR models with more conservative 95 % credible intervals. We did not identify consistent associations of gestational OPE metabolite concentrations with the ponderal index and weekly growth rates. In this cohort, exposure to OPEs during gestation was associated with altered infant anthropometry at 4 weeks after birth.

Practical aspects of Bayesian multivariate meta-analysis

Multivariate meta-analysis is a mostly used approach when multivariate results of several studies are pooled together. The multivariate model of random effects provides a tool to perform the multivariate meta-analysis in practice. In this paper, we discuss Bayesian inference procedures derived for the multivariate model of random effects when the model parameters are endowed with two non-informative priors: the Berger-Bernardo reference prior and the Jeffreys prior. Moreover, two Metropolis-Hastings algorithms are presented, and their convergence properties are analysed via simulations.

Stochastic Convergence Rates and Applications of Adaptive Quadrature in Bayesian Inference

We provide the first stochastic convergence rates for a family of adaptive quadrature rules used to normalize the posterior distribution in Bayesian models. Our results apply to the uniform relative error in the approximate posterior density, the coverage probabilities of approximate credible sets, and approximate moments and quantiles, therefore, guaranteeing fast asymptotic convergence of approximate summary statistics used in practice. The family of quadrature rules includes adaptive Gauss-Hermite quadrature, and we apply this rule in two challenging low-dimensional examples. Further, we demonstrate how adaptive quadrature can be used as a crucial component of a modern approximate Bayesian inference procedure for high-dimensional additive models. The method is implemented and made publicly available in the aghq package for the R language, available on CRAN. Supplementary materials for this article are available online.

Bias-Corrected Maximum Likelihood Estimation and Bayesian Inference for the Process Performance Index Using Inverse Gaussian Distribution

In this study, the estimation methods of bias-corrected maximum likelihood (BCML), bootstrap BCML (B-BCML) and Bayesian using Jeffrey’s prior distribution were proposed for the inverse Gaussian distribution with small sample cases to obtain the ML and Bayes estimators of the model parameters and the process performance index based on the lower specification process performance index. Moreover, an approximate confidence interval and the highest posterior density interval of the process performance index were established via the delta and Bayesian inference methods, respectively. To overcome the computational difficulty of sampling from the posterior distribution in Bayesian inference, the Markov chain Monte Carlo approach was used to implement the proposed Bayesian inference procedures. Monte Carlo simulations were conducted to evaluate the performance of the proposed BCML, B-BCML and Bayesian estimation methods. An example of the active repair times for an airborne communication transceiver is used for illustration.

Bayesian inference for random field parameters with a goal-oriented quality control of the PGD forward model’s accuracy

Numerical models built as virtual-twins of a real structure (digital-twins) are considered the future of monitoring systems. Their setup requires the estimation of unknown parameters, which are not directly measurable. Stochastic model identification is then essential, which can be computationally costly and even unfeasible when it comes to real applications. Efficient surrogate models, such as reduced-order method, can be used to overcome this limitation and provide real time model identification. Since their numerical accuracy influences the identification process, the optimal surrogate not only has to be computationally efficient, but also accurate with respect to the identified parameters. This work aims at automatically controlling the Proper Generalized Decomposition (PGD) surrogate’s numerical accuracy for parameter identification. For this purpose, a sequence of Bayesian model identification problems, in which the surrogate’s accuracy is iteratively increased, is solved with a variational Bayesian inference procedure. The effect of the numerical accuracy on the resulting posteriors probability density functions is analyzed through two metrics, the Bayes Factor (BF) and a criterion based on the Kullback-Leibler (KL) divergence. The approach is demonstrated by a simple test example and by two structural problems. The latter aims to identify spatially distributed damage, modeled with a PGD surrogate extended for log-normal random fields, in two different structures: a truss with synthetic data and a small, reinforced bridge with real measurement data. For all examples, the evolution of the KL-based and BF criteria for increased accuracy is shown and their convergence indicates when model refinement no longer affects the identification results.

Evaluation of longitudinal 13 C isotope ratio mass spectrometric data in antidoping analysis.

The detection of testosterone and its precursors' abuse in antidoping sports analysis is based on the longitudinal evaluation of markers of the urinary endogenous steroid profile. A Bayesian statistical approach is applied, allowing the establishment of credible intervals of the selected parameters for every athlete. Samples showing values outside the acceptable boundaries are selected for additional confirmation by isotope ratio mass spectrometric (IRMS) analysis. The alterations of the IRMS values last longer than the alterations of the steroid profile. Then the application of IRMS to a larger number of samples, at a screening level, would presumably allow detection of additional positive cases. The steroid profile and IRMS data can be treated using the same Bayesian inference procedure. In nonsports population, we have demonstrated the stability of IRMS data. In this work, we studied the variability of these data in real conditions, in samples collected on athletes subjected to antidoping analyses over the years. The data obtained confirmed previous observations and the applicability of the proposed approach. The results of cases where confounding factors of the steroid profile were reported are discussed, showing that in most of the cases no significant changes are observed over the absolute delta values. Changes in diet may significantly change the absolute delta values but not the ones relative to endogenous reference compounds. Finally, a case that could have been evaluated as normal with the current approach without a thorough review of the data was detected as positive by the proposed approach, demonstrating the benefit of its application.

A Multi-Stage Approach for Knowledge-Guided Predictions With Application to Additive Manufacturing

Inspired by sequential additive manufacturing operations, we consider prediction tasks arising in processes that comprise of sequential sub-operations and propose a multi-stage inference procedure that exploits prior knowledge of the operational sequence. Our approach decomposes a data-driven model into several easier problems each corresponding to a sub-operation and then introduces a Bayesian inference procedure to quantify and propagate uncertainty across operational stages. We also complement our model with an approach to incorporate physical knowledge of the output of a sub-operation which is often more practical in reality relative to understanding the physics of the entire process. Comprehensive simulations and two case studies on additive manufacturing show that the proposed framework provides well-quantified uncertainties and superior predictive accuracy compared to a single-stage predictive approach. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by sequential operations that often occur in manufacturing processes. For example, several additive manufacturing processes consist of multiple sequential steps, e.g., printing, washing, and curing in stereolithography, or printing, debinding, and sintering in binder jetting. In such settings, a complex data-driven model that blindly throws all given data into a single predictive model might not be optimal. To this end, we propose a multi-stage inference procedure that decomposes the problem into easier sub-problem using the prior knowledge of the operational sequence, and propagates uncertainty across stages using Bayesian neural networks. Here we note that even if sequential operations are not existent in reality, one may conceptually decompose a complex system into simpler pieces and exploit our procedure. Also, our approach is able to incorporate physical knowledge of the output of a sub-operation.

Bayesian Low-Rank Interpolative Decomposition for Complex Datasets

In this paper, we introduce a probabilistic model for learning interpolative decomposition (ID), which is commonly used for feature selection, low-rank approximation, and identifying hidden patterns in data, where the matrix factors are latent variables associated with each data dimension. Prior densities with support on the specified subspace are used to address the constraint for the magnitude of the factored component of the observed matrix. Bayesian inference procedure based on Gibbs sampling is employed. We evaluate the model on a variety of real-world datasets including CCLE EC50, CCLE IC50, CTRP EC50, and MovieLens 100K datasets with different sizes, and dimensions, and show that the proposed Bayesian ID GBT and GBTN models lead to smaller reconstructive errors compared to existing randomized approaches.

Reducing bias due to misclassified exposures using instrumental variables

AbstractExposures are often misclassified in observational studies. Any analysis that does not make proper adjustments for misclassification may result in biased estimates of model parameters, resulting in distorted inference. Settings where a multicategory exposure variable has more than two nominal categories or where no validation data are available to assess misclassification probabilities are common in practice but seldom considered in the literature. This article presents a novel method of analyzing cohort data with a misclassified, multicategory exposure variable and a binary response variable that uses instrumental variables in lieu of a validation dataset. First, a sufficient condition is obtained for model identifiability. Then, methods for model estimation and inference are proposed after adopting a sufficient condition for identifiability. We consider a variational Bayesian inference procedure aided by automatic differentiation along with Markov chain Monte Carlo‐based computation. Operating characteristics of the proposed methods are assessed through simulation studies. For the purpose of illustration, the proposed Bayesian methods are applied to the U.S. breast cancer mortality data sampled from the Surveillance Epidemiology and End Results database, where reported treatment therapy is the misclassified multicategory exposure variable.