Bayesian Setting Research Articles

A Bayesian multivariate model using Hamiltonian Monte Carlo inference to estimate total organic carbon content in shale

The prediction of total organic carbon (TOC) content using geophysical logs is one of the key steps in shale reservoir characterization. Various empirical relations have previously been used for the estimation of TOC content from well-logs; however, uncertainty quantification in the model estimation is often ignored while performing TOC estimation in a deterministic framework. We introduce the problem of TOC estimation in a Bayesian setting with the goal of enhancing the TOC content prediction together with the quantification of the uncertainty in the model prediction. To signify the uncertainty, we draw random samples of model parameters from the posterior distribution by realizing multidimensional stochastic processes within the Hamiltonian Monte Carlo algorithm. The posterior model for the variables that influence TOC estimation is conditioned on the available well-log observations and is further defined by a priori and likelihood distributions. We demonstrate examples of applications of this approach to estimate the TOC content on two real field data sets from the well-known Devonian Duvernay shale of Western Canada and the Silurian shale of the Ahnet Basin. The accuracy in the estimation is arbitrated by comparing the prediction results with those obtained using the two most widely used empirical models. Corroborating the results by the laboratory-measured TOC contents demonstrate that the Bayesian approach offers a more reliable and better confidence in predictions when compared with the empirical models, as it provides additional information on the prediction uncertainty. Finally, the implications of the present approach are derived in terms of depositional environments to characterize the high TOC content zone in the studied organic shale formations.

A hierarchical Bayesian approach to dynamic ordinary differential equations modeling for repeated measures data on wheat growth

Experimental data collected on growth and development of plants over a growing season are typically analyzed using a linear mixed model, analogous to a hierarchical linear model in a Bayesian setting. Alternative modeling approaches for repeated measures data involve non-linear models such as logistic regression and ecophysiological dynamic models based on a system of ordinary differential equations (ODE). Yet, current implementations of ODE models are mostly deterministic in nature, which negates recognition of uncertainty in the data generation process and thus impairs inference and prediction. The primary objective of this study was to demonstrate the use of a dynamic ODE model within a Bayesian framework to make stochastic inference on system-level parameters. A secondary objective was to compare the predictive performance of an ODE model relative to methodologies more commonly used for repeated measures data from designed experiments, namely hierarchical linear models and hierarchical non-linear models. Using a hierarchical Bayesian implementation, we fit all three types of models to data on leaf area index (LAI) and biomass from a winter wheat dataset. In the context of this application, none of the modeling approaches clearly outperformed any other in terms of goodness of fit or prediction accuracy as indicated by similar posterior median values for root mean squared error (RMSE), Willmott’s agreement index (d), and Nash-Sutcliffe efficiency (NSE). The prediction statistics for the ODE, linear, and non-linear models respectively, were: RMSEp of 1.38, 1.14, and 1.19; dp of 0.91, 0.93, and 0.93; and NSEp of 0.69, 0.78, and 0.76 for LAI and RMSEp of 274.08, 253.04, and 207.63; dp of 0.95, 0.95, and 0.97; and NSEp of 0.82, 0.84, and 0.89 for biomass. The dynamic ODE model enabled biologically meaningful system-level inferences relevant to the research questions that were not possible when using the hierarchical linear or non-linear modeling approaches.

Inductive risk: does it really refute value-freedom?

The argument from inductive risk is considered to be one of the strongest challenges for value-free science. A great part of its appeal lies in the idea that even an ideal epistemic agent—the “perfect scientist” or “scientist qua scientist”—cannot escape inductive risk. In this paper, I scrutinize this ambition by stipulating an idealized Bayesian decision setting. I argue that inductive risk does not show that the “perfect scientist” must, descriptively speaking, make non-epistemic value-judgements, at least not in a way that undermines the value-free ideal. However, the argument is more successful in showing that there are cases where the “perfect scientist” should, normatively speaking, use non-epistemic values. I also show that this is possible without creating problems of illegitimate prescription and wishful thinking. Thus, while inductive risk does not refute value-freedom completely, it still represents a powerful critique of value-free science.

Fourier Representations for Black-Box Optimization over Categorical Variables

Optimization of real-world black-box functions defined over purely categorical variables is an active area of research. In particular, optimization and design of biological sequences with specific functional or structural properties have a profound impact in medicine, materials science, and biotechnology. Standalone search algorithms, such as simulated annealing (SA) and Monte Carlo tree search (MCTS), are typically used for such optimization problems. In order to improve the performance and sample efficiency of such algorithms, we propose to use existing methods in conjunction with a surrogate model for the black-box evaluations over purely categorical variables. To this end, we present two different representations, a group-theoretic Fourier expansion and an abridged one-hot encoded Boolean Fourier expansion. To learn such representations, we consider two different settings to update our surrogate model. First, we utilize an adversarial online regression setting where Fourier characters of each representation are considered as experts and their respective coefficients are updated via an exponential weight update rule each time the black box is evaluated. Second, we consider a Bayesian setting where queries are selected via Thompson sampling and the posterior is updated via a sparse Bayesian regression model (over our proposed representation) with a regularized horseshoe prior. Numerical experiments over synthetic benchmarks as well as real-world RNA sequence optimization and design problems demonstrate the representational power of the proposed methods, which achieve competitive or superior performance compared to state-of-the-art counterparts, while improving the computation cost and/or sample efficiency, substantially.

Power of Bonus in Pricing for Crowdsourcing

We consider a simple form of pricing for a crowdsourcing system, where pricing policy is published a priori, and workers then decide their task acceptance. Such a pricing form is widely adopted in practice for its simplicity, e.g., Amazon Mechanical Turk, although additional sophistication to pricing rule can enhance budget efficiency. With the goal of designing efficient and simple pricing rules, we study the impact of the following two design features in pricing policies: (i) personalization tailoring policy worker-by-worker and (ii) bonus payment to qualified task completion. In the Bayesian setting, where the only prior distribution of workers' profiles is available, we first study the Price of Agnosticism (PoA) that quantifies the utility gap between personalized and common pricing policies. We show that PoA is bounded within a constant factor under some mild conditions, and the impact of bonus is essential in common pricing. These analytic results imply that complex personalized pricing can be replaced by simple common pricing once it is equipped with a proper bonus payment. To provide insights on efficient common pricing, we then study the efficient mechanisms of bonus payment for several profile distribution regimes which may exist in practice. We provide primitive experiments on Amazon Mechanical Turk, which support our analytical findings[5].

Fitting Structural Equation Models via Variational Approximations

Structural equation models are commonly used to capture the relationship between sets of observed and unobservable variables. Traditionally these models are fitted using frequentist approaches, but recently researchers and practitioners have developed increasing interest in Bayesian inference. In Bayesian settings, inference for these models is typically performed via Markov chain Monte Carlo methods, which may be computationally intensive for models with a large number of manifest variables or complex structures. Variational approximations can be a fast alternative; however, they have not been adequately explored for this class of models. We develop a mean field variational Bayes approach for fitting elemental structural equation models and demonstrate how bootstrap can considerably improve the variational approximation quality. We show that this variational approximation method can provide reliable inference while being significantly faster than Markov chain Monte Carlo methods.

Designing Incentives for Heterogeneous Researchers

A principal (e.g., the US government) contracts with a researcher with unknown costs (e.g., a vaccine developer) to conduct a costly experiment. This contracting problem has a novel feature that captures the difference between the form of an experiment and the strength of its results: researchers face a problem of information design rather than optimal effort. Using a novel comparative static for Bayesian persuasion settings, I characterize the optimal contract and show how experimentation is distorted by the need to screen researchers. Moreover, I show that there is no loss from contracting on the experiment’s result rather than the experiment itself.

Evaluation of the Radiobiological Models Predicting the Radiation-Induced Hypothyroidism in the Partially Irradiated Thyroid Gland of Patients with Breast Cancer

Background: Radiation-induced hypothyroidism (RHT) is one of the side effects that might have an impact on the quality of life of patients with breast cancer treated with radiotherapy. Objectives: The aim of the current study was to evaluate the performances of the Lyman-Kutcher-Burman (LKB) and Log-Logistic models in the prediction of hypothyroidism (HT) as well as the estimation of the model parameters for the incidence of RHT among patients with breast cancer. Methods: Fifty-two patients treated with radiation therapy (RT) for breast cancer were prospectively evaluated. Patients' serum samples [tri-iodothyronine, thyroxine, thyroid-stimulating hormone (TSH), free triiodothyronine, and free thyroxine] were measured before RT and also at a regular time interval until 1 year after the completion of RT. For each patient, dose-volume histograms (DVHs) of the thyroid gland were derived from their treatment planning dataset. Patients whose TSH levels were higher than normal with a decrease in FT4 levels were considered as cases with RHT. The LKB and Log-Logistic radiobiological models were evaluated by comparing them with the resultant follow-up data. The parameters for radiobiological models have been deduced by fitting the models to the follow-up data. The models were fitted in a Bayesian setting and compared according to the widely applicable information criterion (WAIC). Results: Twenty-one (40%) patients developed RHT at a follow-up of 1 year after the end of radiation treatment. The fitted values of D50 for the LKB and Log-Logistic models were 37.71 and 25.50 Gy, respectively for the partially irradiated thyroid of patients with breast cancer. The mean time to the incidence of RHT was obtained at 6.7 months in the studied group. Conclusions: A volumetric effect was found for the thyroid gland in the implemented normal tissue complication probability models. Compared to the follow-up data, the Log-Logistic model was ranked as the best model for predicting the rate of RHT in patients with breast cancer.

Prepare for the Worst, Hope for the Best: Active Robust Learning On Distributions.

In recent years, many learning systems have been developed for higher level forms of data, such as learning on distributions in which each example itself is a distribution. This article proposes active robust learning on distributions. In learning on distributions, there is no access to distributions themselves but rather access is through a sample drawn from a distribution. Therefore, similar to robust learning, any estimates of examples are inexact. In order to address these difficulties, we provide an upper bound on the risk of the classifier in the next stage of active learning, where the size of the labeled dataset increases. Based on this upper bound, we propose probabilistic minimax active learning (PMAL) as a general multiclass active learning method that is easy to use in many Bayesian settings, which provably selects an example with knowledge of its label minimizing the expected risk. We present an efficient approximation of the objective with a known error bound to deal with the intractability of the proposed method for active robust learning. Here, we face a nonconvex problem, which we solve by means of a related convex problem with a bound on the norm of the difference between their solutions. To utilize the information about the estimates of distributions, we propose active robust learning on the distributions method based on learning the kernel embedding of distributions by a recent Bayesian method. The experiments demonstrate the effectiveness of the resulting method on a set of synthetic and real-world distributional datasets.

Accelerated parallel non-conjugate sampling for Bayesian non-parametric models

Inference of latent feature models in the Bayesian nonparametric setting is generally difficult, especially in high dimensional settings, because it usually requires proposing features from some prior distribution. In special cases, where the integration is tractable, we can sample new feature assignments according to a predictive likelihood. We present a novel method to accelerate the mixing of latent variable model inference by proposing feature locations based on the data, as opposed to the prior. First, we introduce an accelerated feature proposal mechanism that we show is a valid MCMC algorithm for posterior inference. Next, we propose an approximate inference strategy to perform accelerated inference in parallel. A two-stage algorithm that combines the two approaches provides a computationally attractive method that can quickly reach local convergence to the posterior distribution of our model, while allowing us to exploit parallelization.

Bayesian Inversion with Open-Source Codes for Various One-Dimensional Model Problems in Computational Mechanics

The complexity of many problems in computational mechanics calls for reliable programming codes and accurate simulation systems. Typically, simulation responses strongly depend on material and model parameters, where one distinguishes between backward and forward models. Providing reliable information for the material/model parameters, enables us to calibrate the forward model (e.g., a system of PDEs). Markov chain Monte Carlo methods are efficient computational techniques to estimate the posterior density of the parameters. In the present study, we employ Bayesian inversion for several mechanical problems and study its applicability to enhance the model accuracy. Seven different boundary value problems in coupled multi-field (and multi-physics) systems are presented. To provide a comprehensive study, both rate-dependent and rate-independent equations are considered. Moreover, open source codes (https://doi.org/10.5281/zenodo.6451942) are provided, constituting a convenient platform for future developments for, e.g., multi-field coupled problems. The developed package is written in MATLAB and provides useful information about mechanical model problems and the backward Bayesian inversion setting.

Bayesian Kernel Two-Sample Testing

In modern data analysis, nonparametric measures of discrepancies between random variables are particularly important. The subject is well-studied in the frequentist literature, while the development in the Bayesian setting is limited where applications are often restricted to univariate cases. Here, we propose a Bayesian kernel two-sample testing procedure based on modeling the difference between kernel mean embeddings in the reproducing kernel Hilbert space using the framework established by Flaxman et al. The use of kernel methods enables its application to random variables in generic domains beyond the multivariate Euclidean spaces. The proposed procedure results in a posterior inference scheme that allows an automatic selection of the kernel parameters relevant to the problem at hand. In a series of synthetic experiments and two real data experiments (i.e., testing network heterogeneity from high-dimensional data and six-membered monocyclic ring conformation comparison), we illustrate the advantages of our approach. Supplementary materials for this article are available online.

Wool scoured colour: Heritability, genetic and phenotypic correlations with wool traits in Corriedale sheep

Scoured wool colour determines the potenti al colour range and dye-ability of wool tops. Improvement of scoured wool colour of fleeces from Corriedale sheep through multi-trait breeding programmes is currently not possible due to limited existing information on its heritability and genetic correlations with other fleece traits. The present study aimed to determine genetic parameters of scoured wool colour traits, with emphasis on their genetic and phenotypic correlations with objectively and visually assessed fleece traits. Data from 1181 hoggets of two experimental flocks were used. Heritabilities were estimated for scoured wool yellowness (Y-Z) and scoured wool brightness (Y), greasy fleece weight (GFW) and clean fleece weight (CFW), wool yield (WY), mean fibre diameter (MFD), coefficient of variation of fibre diameter (CVD), staple length (SL), staple strength (SS) and a range of visual wool traits (greasy wool colour, wool character, handle, staple formation, dust penetration and cross-linking). Trivariate statistical models under a Bayesian setting were used, including the environmental effects of flock, sex, litter size and climatic variables (accumulated precipitation, maximum temperature and relative humidity) and the animal genetic effect, Y-Z was moderately heritable (0.22 ± 0.06) and should respond to selection. Its genetic correlations with GFW, CFW and SL were significant, positive (unfavourable) and moderate (0.43 ± 0.15, 0.56 ± 0.14, and 0.44 ± 0.14, respectively). There was a positive (favourable) association between MFD and Y-Z (0.25 ± 0.17). The genetic correlation between Y-Z and greasy wool colour was 0.70 ± 0.13, although this trait showed a low heritability value (0.13 ± 0.04). Wool brightness showed low heritability (0.10 ± 0.04) and only appeared to be genetically correlated with SL and wool character (0.56 ± 0.17 and 0.39 ± 0.22, respectively). Phenotypic correlations of Y-Z and Y with wool production and quality traits were all less than 0.20 in size. Overall, this study reports new genetic parameters for wool colour and their associations with fleece traits in Corriedale sheep. Our findings suggest that it could be possible to select against wool yellowness, with some caution, given the possible detrimental effects on economically important wool traits, especially fleece weight and staple length. More investigation is warranted. Indirect selection via greasy wool colour is an alternative approach, but the responses would be slower. Easy and cheap measurement should be balanced against the size of the response from direct selection versus indirect selection.

Fundamental Limits in Bayesian Thermometry and Attainability via Adaptive Strategies.

We investigate the limits of thermometry using quantum probes at thermal equilibrium within the Bayesian approach. We consider the possibility of engineering interactions between the probes in order to enhance their sensitivity, as well as feedback during the measurement process, i.e., adaptive protocols. On the one hand, we obtain an ultimate bound on thermometry precision in the Bayesian setting, valid for arbitrary interactions and measurement schemes, which lower bounds the error with a quadratic (Heisenberg-like) scaling with the number of probes. We develop a simple adaptive strategy that can saturate this limit. On the other hand, we derive a no-go theorem for nonadaptive protocols that does not allow for better than linear (shot-noise-like) scaling even if one has unlimited control over the probes, namely, access to arbitrary many-body interactions.

Assessement of Enterprise Interoperability Maturity Level through Generative and Recognition Mo

In a globalized and networked society, enterprise interoperability is a key factor of success for enterprises in their effort to maximize their own added values and to exploit the market opportunities. The sustainable enterprise interoperability is a continuous challenge of the networked collaborative environment. By making business decisions, managers have to take into account the maturity level of their own enterprise and of others’ with whom they get involved into businesses. Maturity level of enterprise interoperability has been defined by the Framework for Enterprise Interoperability (FEI), standardized by CEN EN ISO 11354. In this paper, we propose a novel approach to assess maturity levels of enterprise interoperability (MLEI) through latent factor analysis (LFA) and generative and recognition models applied to the categories and features defined by FEI. Given an enterprise interoperability maturity matrix we have trained a stochastic neural network, namely Restricted Bolzmann Machine (RBM) to learn the MLEI. Our research seeks to answer the following questions: whether the maturity level assessed by evaluators correlate with the maturity levels recognized by RBM trained in a supervised learning representation, and how to model recognition matrix of MLEI by using maturity level correlations between observed performances (inputs) and latent or hidden factors that influence the correct assessment. We considered a maturity level correlation matrix representing the enterprise features as defined in FEI in addition to a set of latent factors, representing the type of maturity level of each individual enterprise. Our proposal is based on a generative and a recognition model using deterministic non-linear functions in a Bayesian setting. The model has been tested on artificial data by training a RBM. Experiments on artificial data sets of enterprises proved that our proposal is a reliable approach that can be further developed into a methodology and extended for the design of adaptive learning agents. In the perspective of the Future Internet, such agents may successfully assist human evaluators in the tedious and time consuming process of the assessment of MLEI in real settings.

Probabilistic inversions of electrical resistivity tomography data with a machine learning‐based forward operator

ABSTRACTCasting a geophysical inverse problem into a Bayesian setting is often discouraged by the computational workload needed to run many forward modelling evaluations. Here we present probabilistic inversions of electrical resistivity tomography data in which the forward operator is replaced by a trained residual neural network that learns the non‐linear mapping between the resistivity model and the apparent resistivity values. The use of this specific architecture can provide some advantages over standard convolutional networks as it mitigates the vanishing gradient problem that might affect deep networks. The modelling error introduced by the network approximation is properly taken into account and propagated onto the estimated model uncertainties. One crucial aspect of any machine learning application is the definition of an appropriate training set. We draw the models forming the training and validation sets from previously defined prior distributions, while a finite element code provides the associated datasets. We apply the approach to two probabilistic inversion frameworks: A Markov chain Monte Carlo algorithm is applied to synthetic data, while an ensemble‐based algorithm is employed for the field measurements. For both the synthetic and field tests, the outcomes of the proposed method are benchmarked against the predictions obtained when the finite element code constitutes the forward operator. Our experiments illustrate that the network can effectively approximate the forward mapping even when a relatively small training set is created. The proposed strategy provides a forward operator that is three orders of magnitude faster than the accurate but computationally expensive finite element code. Our approach also yields the most likely solutions and uncertainty quantifications comparable to those estimated when the finite element modelling is employed. The presented method allows solving the Bayesian electrical resistivity tomography with a reasonable computational cost and limited hardware resources.

A comparison of nonlinear extensions to the ensemble Kalman filter: Gaussian anamorphosis and two-step ensemble filters.

Ensemble Kalman filters are based on a Gaussian assumption, which can limit their performance in some non-Gaussian settings. This paper reviews two nonlinear, non-Gaussian extensions of the Ensemble Kalman Filter: Gaussian anamorphosis (GA) methods and two-step updates, of which the rank histogram filter (RHF) is a prototypical example. GA-EnKF methods apply univariate transforms to the state and observation variables to make their distribution more Gaussian before applying an EnKF. The two-step methods use a scalar Bayesian update for the first step, followed by linear regression for the second step. The connection of the two-step framework to the full Bayesian problem is made, which opens the door to more advanced two-step methods in the full Bayesian setting. A new method for the first part of the two-step framework is proposed, with a similar form to the RHF but a different motivation, called the ‘improved RHF’ (iRHF). A suite of experiments with the Lorenz-‘96 model demonstrate situations where the GA-EnKF methods are similar to EnKF, and where they outperform EnKF. The experiments also strongly support the accuracy of the RHF and iRHF filters for nonlinear and non-Gaussian observations; these methods uniformly beat the EnKF and GA-EnKF methods in the experiments reported here. The new iRHF method is only more accurate than RHF at small ensemble sizes in the experiments reported here.

A Bayesian Spatio-Network Model for Multiple Adolescent Adverse Health Behaviours

Abstract The use of alcohol, cigarettes and marijuana among adolescents are major public health concerns, and a number of epidemiological studies have been conducted to understand the drivers of these individual health behaviours. However, there is no literature that jointly models these health behaviours with the aim of understanding the relative importance of individual factors, friendship effects and spatial effects in determining the prevalence of alcohol, cigarette and marijuana use among adolescents. To address this gap in the literature, we propose a novel multivariate spatio-network model for jointly modelling all three of these behaviours, with inference conducted in a Bayesian setting using Markov chain Monte Carlo simulation. The model is motivated by survey data from five schools in Los Angeles, California, and the results indicate the important roles that individual factors and friendship networks play in driving the uptake of these health behaviours.

Bayesian agency: Linear versus tractable contracts

We study principal-agent problems in which a principal commits to an outcome-dependent payment scheme (a.k.a. contract) so as to induce an agent to take a costly, unobservable action. We relax the assumption that the principal perfectly knows the agent by considering a Bayesian setting where the agent's type is unknown and randomly selected according to a given probability distribution, which is known to the principal. Each agent's type is characterized by her own action costs and action-outcome distributions. In the literature on non-Bayesian principal-agent problems, considerable attention has been devoted to linear contracts, which are simple, pure-commission payment schemes that still provide nice approximation guarantees with respect to principal-optimal (possibly non-linear) contracts. While in non-Bayesian settings an optimal contract can be computed efficiently, this is no longer the case for our Bayesian principal-agent problems. This further motivates our focus on linear contracts, which can be optimized efficiently given their single-parameter nature. Our goal is to analyze the properties of linear contracts in Bayesian settings, in terms of approximation guarantees with respect to optimal contracts and general tractable contracts (i.e., efficiently-computable ones). First, we study the approximation guarantees of linear contracts with respect to optimal ones, showing that the former suffer from a multiplicative loss that grows linearly in the number of agent's types. Nevertheless, we prove that linear contracts can still provide a constant multiplicative approximation ρ of the optimal principal's expected utility, though at the expense of an exponentially-small additive loss 2−Ω(ρ). Then, we switch to tractable contracts, showing that, surprisingly, linear contracts perform well among them. In particular, we prove that it is NP-hard to design a contract providing a multiplicative loss sublinear in the number of agent's types, while the same holds for contracts that provide a constant multiplicative approximation ρ at the expense of an additive loss 2−ω(ρ). We conclude by showing that, in Bayesian principal-agent problems, an optimal contract can be computed efficiently if we fix either the number of agent's types or the number of outcomes.

ALTERNATIVE ROBUST ESTIMATORS FOR PARAMETERS OF THE LINEAR REGRESSION MODEL

This paper considers parameter estimation of the linear regression model with Ramsay-Novick (RN) distributed errors, focusing on its use as an aid to robustness. Positioning within the class of heavy tailed distributions, RN distribution can be defined as the modification of unbounded influence function of a non-robust density so that it has more resistance to outliers. Potential use of this robust density have so far been assessed in Bayesian settings on real data examples and there is a lack of performance assessment for finite samples in classical approach. This study therefore explores its robustness properties when used as error distribution in comparison to normal as well as other alternating heavy-tailed distributions like Laplace and Student-t. An extensive simulation study was conducted for this purpose under different settings of sample size, model parameters and outlier percantages. An efficient data generation of this distribution through random-walk Metropolis algorithm is here also suggested. The results were supported by a real world application on famously known as Brownlee’s stack loss plant data.