Conjugate Prior Research Articles

Detection and quantification of introgression using Bayesian inference based on conjugate priors.

Introgression (the flow of genes between species) is a major force structuring the evolution of genomes, potentially providing raw material for adaptation. Here, we present a versatile Bayesian model selection approach for detecting and quantifying introgression, df-BF, that builds upon the recently published distance-based df statistic. Unlike df, df-BF accounts for the number of variant sites within a genomic region. The underlying model parameter of our df-BF method, here denoted as dfθ, accurately quantifies introgression, and the corresponding Bayes Factors (df-BF) enables weighing the strength of evidence for introgression. To ensure fast computation, we use conjugate priors with no need for computationally demanding MCMC iterations. We compare our method with other approaches including df, fd, Dp, and Patterson's D using a wide range of coalescent simulations. Furthermore, we showcase the applicability of df-BF and dfθ using whole-genome mosquito data. Finally, we integrate the new method into the powerful genomics R-package PopGenome. The presented methods are implemented within the R-package PopGenome (https://github.com/pievos101/PopGenome) and the simulation as the application results can be reproduced from the source code available from a dedicated GitHub repository (https://github.com/pievos101/Introgression-Simulation).

Bayesian Lower and Upper Estimates for Ether Option Prices with Conditional Heteroscedasticity and Model Uncertainty

This paper aims to leverage Bayesian nonlinear expectations to construct Bayesian lower and upper estimates for prices of Ether options, that is, options written on Ethereum, with conditional heteroscedasticity and model uncertainty. Specifically, a discrete-time generalized conditional autoregressive heteroscedastic (GARCH) model is used to incorporate conditional heteroscedasticity in the logarithmic returns of Ethereum, and Bayesian nonlinear expectations are adopted to introduce model uncertainty, or ambiguity, about the conditional mean and volatility of the logarithmic returns of Ethereum. Extended Girsanov’s principle is employed to change probability measures for introducing a family of alternative GARCH models and their risk-neutral counterparts. The Bayesian credible intervals for “uncertain” drift and volatility parameters obtained from conjugate priors and residuals obtained from the estimated GARCH model are used to construct Bayesian superlinear and sublinear expectations giving the Bayesian lower and upper estimates for the price of an Ether option, respectively. Empirical and simulation studies are provided using real data on Ethereum in AUD. Comparisons with a model incorporating conditional heteroscedasticity only and a model capturing ambiguity only are presented.

Exploring Estimation Procedures for Reducing Dimensionality in Psychological Network Modeling.

To understand psychological data, it is crucial to examine the structure and dimensions of variables. In this study, we examined alternative estimation algorithms to the conventional GLASSO-based exploratory graph analysis (EGA) in network psychometric models to assess the dimensionality structure of the data. The study applied Bayesian conjugate or Jeffreys' priors to estimate the graphical structure and then used the Louvain community detection algorithm to partition and identify groups of nodes, which allowed the detection of the multi- and unidimensional factor structures. Monte Carlo simulations suggested that the two alternative Bayesian estimation algorithms had comparable or better performance when compared with the GLASSO-based EGA and conventional parallel analysis (PA). When estimating the multidimensional factor structure, the analytically based method (i.e., EGA.analytical) showed the best balance between accuracy and mean biased/absolute errors, with the highest accuracy tied with EGA but with the smallest errors. The sampling-based approach (EGA.sampling) yielded higher accuracy and smaller errors than PA; lower accuracy but also lower errors than EGA. Techniques from the two algorithms had more stable performance than EGA and PA across different data conditions. When estimating the unidimensional structure, the PA technique performed the best, followed closely by EGA, and then EGA.analytical and EGA.sampling. Furthermore, the study explored four full Bayesian techniques to assess dimensionality in network psychometrics. The results demonstrated superior performance when using Bayesian hypothesis testing or deriving posterior samples of graph structures under small sample sizes. The study recommends using the EGA.analytical technique as an alternative tool for assessing dimensionality and advocates for the usefulness of the EGA.sampling method as a valuable alternate technique. The findings also indicated encouraging results for extending the regularization-based network modeling EGA method to the Bayesian framework and discussed future directions in this line of work. The study illustrated the practical application of the techniques to two empirical examples in R.

Bayesian inference of sample-specific coexpression networks.

Gene regulatory networks (GRNs) are effective tools for inferring complex interactions between molecules that regulate biological processes and hence can provide insights into drivers of biological systems. Inferring coexpression networks is a critical element of GRN inference, as the correlation between expression patterns may indicate that genes are coregulated by common factors. However, methods that estimate coexpression networks generally derive an aggregate network representing the mean regulatory properties of the population and so fail to fully capture population heterogeneity. Bayesian optimized networks obtained by assimilating omic data (BONOBO) is a scalable Bayesian model for deriving individual sample-specific coexpression matrices that recognizes variations in molecular interactions across individuals. For each sample, BONOBO assumes a Gaussian distribution on the log-transformed centered gene expression and a conjugate prior distribution on the sample-specific coexpression matrix constructed from all other samples in the data. Combining the sample-specific gene coexpression with the prior distribution, BONOBO yields a closed-form solution for the posterior distribution of the sample-specific coexpression matrices, thus allowing the analysis of large data sets. We demonstrate BONOBO's utility in several contexts, including analyzing gene regulation in yeast transcription factor knockout studies, the prognostic significance of miRNA-mRNA interaction in human breast cancer subtypes, and sex differences in gene regulation within human thyroid tissue. We find that BONOBO outperforms other methods that have been used for sample-specific coexpression network inference and provides insight into individual differences in the drivers of biological processes.

A note on Bayesian inference for the bivariate pseudo-exponential model

In this present work, we discuss the Bayesian inference for the Bivariate Pseudo-exponential distribution. Initially, we assume independent gamma priors and Pseudo-gamma priors for the Pseudo-exponential parameters. Possible conjugacy is investigated in both full and sub-models. In some special sub-cases, conjugate priors can be identified. We are primarily interested in deriving the posterior means for each prior assumed and comparing each of the posterior means with the maximum likelihood estimators. Finally, all the Bayesian analyses are illustrated with a simulation study to demonstrate the performance of Bayesian parameter estimates using a variety of informative and non-informative priors, also illustrated with real-life applications.

A variational Bayesian based robust filter for unknown measurement bias and inaccurate noise statistics

In many practical fields, the unknown time-varying measurement biases (additive and multiplicative bias) and heavy-tailed measurement noise caused by some unpredictable anomalous behaviors may degrade the performance of conventional Kalman filter seriously. To solve the state estimation problem of systems with time-varying measurement biases and heavy-tailed measurement noise, this paper proposes a new variational Bayesian (VB) based robust filter. Firstly, the non-Gaussian measurement likelihood probability density function (ML-PDF) with multiplicative and additive measurement bias is built. Then, the conjugate prior distributions for unknown bias and noise scale parameters are selected, and the VB method is utilized to jointly infer the system state, unknown measurement biases and inaccurate measurement noise covariance matrix. Finally, a VB based robust filter is derived and its effectiveness is verified by the numerical simulations.

Beyond conjugacy for chain event graph model selection

Chain event graphs are a family of probabilistic graphical models that generalise Bayesian networks and have been successfully applied to a wide range of domains. Unlike Bayesian networks, these models can encode context-specific conditional independencies as well as asymmetric developments within the evolution of a process. More recently, new model classes belonging to the chain event graph family have been developed for modelling time-to-event data to study the temporal dynamics of a process. However, existing Bayesian model selection algorithms for chain event graphs and its variants rely on all parameters having conjugate priors. This is unrealistic for many real-world applications. In this paper, we propose a mixture modelling approach to model selection in chain event graphs that does not rely on conjugacy. Moreover, we show that this methodology is more amenable to being robustly scaled than the existing model selection algorithms used for this family. We demonstrate our techniques on simulated datasets.

Outcome-guided Bayesian clustering for disease subtype discovery using high-dimensional transcriptomic data

Due to the tremendous heterogeneity of disease manifestations, many complex diseases that were once thought to be single diseases are now considered to have disease subtypes. Disease subtyping analysis, that is the identification of subgroups of patients with similar characteristics, is the first step to accomplish precision medicine. With the advancement of high-throughput technologies, omics data offers unprecedented opportunity to reveal disease subtypes. As a result, unsupervised clustering analysis has been widely used for this purpose. Though promising, the subtypes obtained from traditional quantitative approaches may not always be clinically meaningful (i.e. correlate with clinical outcomes). On the other hand, the collection of rich clinical data in modern epidemiology studies has the great potential to facilitate the disease subtyping process via omics data and to discovery clinically meaningful disease subtypes. Thus, we developed an outcome-guided Bayesian clustering (GuidedBayesianClustering) method to fully integrate the clinical data and the high-dimensional omics data. A Gaussian mixed model framework was applied to perform sample clustering; a spike-and-slab prior was utilized to perform gene selection; a mixture model prior was employed to incorporate the guidance from a clinical outcome variable; and a decision framework was adopted to infer the false discovery rate of the selected genes. We deployed conjugate priors to facilitate efficient Gibbs sampling. Our proposed full Bayesian method is capable of simultaneously (i) obtaining sample clustering (disease subtype discovery); (ii) performing feature selection (select genes related to the disease subtype); and (iii) utilizing clinical outcome variable to guide the disease subtype discovery. The superior performance of the GuidedBayesianClustering was demonstrated through simulations and applications of breast cancer expression data and Alzheimer's disease. An R package has been made publicly available on GitHub to improve the applicability of our method.

Variational robust filter for a class of stochastic systems with false and missing measurements

This paper investigates the state estimation problem for a class of stochastic system under randomly occurring measurement anomalies, and the Kalman filter is combined with variational Bayesian (VB) method to cope with the simultaneous occurrence of false and missing measurements. First, to unify the false and missing measurements into a modified measurement model, a categorical distributed vector is employed to establish a new measurement model including randomly occurring measurement anomalies. Next, the conjugate prior distributions for the unknown measurement anomaly parameters are determined, in which the probabilities of the measurement anomalies are modeled as Dirichlet distribution and the false measurement is described by Gaussian-inverse-Wishart distribution. Then, based on the constructed measurement model and VB inference, a variational robust KF is designed to simultaneously estimate the state and measurement anomaly parameters. Finally, the estimation performance of the proposed filter is illustrated through a simulation example.

Two Bayesian approaches of monitoring mean of Gaussian process using Bayes factor

AbstractThis paper develops two novel process monitoring schemes for the mean of a Gaussian process: the Bayes factor (BF) and the improved Bayes factor (IBF) schemes. Conjugate priors are used to construct the plotting statistics. The performance of the proposed schemes is evaluated in terms of average run length (ARL), standard deviation of run length (SDRL), and several percentiles, and these performance metrics across different hyper‐parameters and various sample sizes are evaluated via Monte Carlo simulations. Both zero‐state and steady‐state out‐of‐control (OOC) performances are investigated comprehensively. The simulation results show that the IBF scheme outperforms the existing Bayesian exponentially weighted moving average (EWMA) schemes under different loss functions in zero‐state. In steady‐state conditions, the IBF scheme outperforms for small shifts. Finally, we present two examples to illustrate the practical application of the proposed schemes.

The Effect of the Prior and the Experimental Design on the Inference of the Precision Matrix in Gaussian Chain Graph Models

AbstractHere, we investigate whether (and how) experimental design could aid in the estimation of the precision matrix in a Gaussian chain graph model, especially the interplay between the design, the effect of the experiment and prior knowledge about the effect. Estimation of the precision matrix is a fundamental task to infer biological graphical structures like microbial networks. We compare the marginal posterior precision of the precision matrix under four priors: flat, conjugate Normal-Wishart, Normal-MGIG and a general independent. Under the flat and conjugate priors, the Laplace-approximated posterior precision is not a function of the design matrix rendering useless any efforts to find an optimal experimental design to infer the precision matrix. In contrast, the Normal-MGIG and general independent priors do allow for the search of optimal experimental designs, yet there is a sharp upper bound on the information that can be extracted from a given experiment. We confirm our theoretical findings via a simulation study comparing (i) the KL divergence between prior and posterior and (ii) the Stein’s loss difference of MAPs between random and no experiment. Our findings provide practical advice for domain scientists conducting experiments to better infer the precision matrix as a representation of a biological network.

Minimally monophyletic genera are the cast-iron building blocks of evolution

Detailed evaluation is provided for the statistical methods intrinsic to interlocking Sequential Bayes analysis, which allows estimation of evidential support for stem-taxon dendrograms charting the macroevolution of taxa. It involves complexity functions, such as fractal evolution, to generate well-supported evolutionary trees. Required are data on trait changes from ancestral species to descendant species, which is facilitated by reduction of large genera to the smallest included monophyletic groups (one inferred ancestral species each). The genus is here defined as the smallest monophyletic unit, which turns out to be monothetic at least for the direct descendant species. The key fact is that the most-recently acquired traits of the single ancestral species are apparently selectively inviolate and passed on without change to each immediate descendant species. The details of sequential Bayesian analysis were clarified by comparing support of the optimal model with summed support of the alternative models. Because analysis is confined to optimal arrangements of only immediate branches from ancestral species to descendant species, conjugate priors were found to operate such that all alternative models are simply one minus the probability of the optimal model. Such analysis demonstrated that the optimum arrangement of ancestor and descendant species leads to high support values for fitting evolutionary theory, comparable to statistical support levels reported for molecular evolutionary trees, and conjugate priors may be assumed for similar model-building. The method is simple, free of special computer analysis, and well-suited to standard taxonomic practice.

Bayesian Semiparametric Longitudinal Inverse-Probit Mixed Models for Category Learning.

Understanding how the adult human brain learns novel categories is an important problem in neuroscience. Drift-diffusion models are popular in such contexts for their ability to mimic the underlying neural mechanisms. One such model for gradual longitudinal learning was recently developed in Paulon et al. (J Am Stat Assoc 116:1114-1127, 2021). In practice, category response accuracies are often the only reliable measure recorded by behavioral scientists to describe human learning. Category response accuracies are, however, often the only reliable measure recorded by behavioral scientists to describe human learning. To our knowledge, however, drift-diffusion models for such scenarios have never been considered in the literature before. To address this gap, in this article, we build carefully on Paulon et al. (J Am Stat Assoc 116:1114-1127, 2021), but now with latent response times integrated out, to derive a novel biologically interpretable class of 'inverse-probit' categorical probability models for observed categories alone. However, this new marginal model presents significant identifiability and inferential challenges not encountered originally for the joint model in Paulon et al. (J Am Stat Assoc 116:1114-1127, 2021). We address these new challenges using a novel projection-based approach with a symmetry-preserving identifiability constraint that allows us to work with conjugate priors in an unconstrained space. We adapt the model for group and individual-level inference in longitudinal settings. Building again on the model's latent variable representation, we design an efficient Markov chain Monte Carlo algorithm for posterior computation. We evaluate the empirical performance of the method through simulation experiments. The practical efficacy of the method is illustrated in applications to longitudinal tone learning studies.

Bayesian multinomial model with inference based on Pólya-Gamma augmentation

Statistical inferences from multi-categorical data are often encountered in many scientific fields. It is customary in the literature to model the probability of occurrence of a certain category on some risk factors. However, this work presents a framework to model the stick-breaking probabilities of a multi-categorical event on risk factors. In addition, we adopted a Pólya-Gamma data augmentation to arrive at a conjugate prior distribution for the model parameters and developed a cheap Gibbs sampling algorithm for model estimation. The proposed model allows for flexible and computationally efficient modeling of the conditional properties of an ordered categorical process and the incorporation of dependencies between the different categories of the outcome. The framework was used to model data derived from the Nigeria Demographic and Health Survey to study the spatial patterns and risk factors of maternal malnutrition in Nigeria. The findings identified the determinant factors and showed heterogeneity in the spatial patterns of maternal malnutrition prevalence across Nigeria. Findings from the analysis could be relevant for designing intervention programs and guiding policy-making, such as in the allocation of limited resources across the country.

A note on conjugate Bayesian estimators of random effects model

The theoretical findings for the Bayes random effects model and the Bayes random effects model with linear constraints on the model coefficients are the contribution of this study. We take into account the random effect model, which includes both fixed and random effects in addition to the experimental error term. We sought to offer a detailed examination of some characteristics of the Bayes and restricted Bayes estimators of the model in addition to applying the Bayesian approach to draw conclusions about the model using a conjugate prior distribution.

Analysis of a new jointly hybrid censored Rayleigh populations

<abstract><p>When a researcher wants to perform a life-test comparison study of items made by two separate lines inside the same institution, joint censoring strategies are particularly important. In this paper, we present a new joint Type-Ⅰ hybrid censoring that enables an experimenter to stop the investigation as soon as a pre-specified number of failures or time is first achieved. In the context of newly censored data, the estimates of the unknown mean lifetimes of two different Rayleigh populations are acquired using maximum likelihood and Bayesian inferential techniques. The normality characteristic of classical estimators is used to offer asymptotic confidence interval bounds for each unknown parameter. Against gamma conjugate priors, the Bayes estimators and related credible intervals are gathered about symmetric and asymmetric loss functions. Since classical and Bayes estimators are acquired in closed form, simulation tests can be easily made to evaluate the effectiveness of the proposed methodologies. The efficiency of the suggested approaches is examined in terms of four metrics, namely: Root mean squared error, average relative absolute bias, average confidence length, and coverage probability. To demonstrate the applicability of the offered approaches to real events, two real applications employing data sets from the engineering area are analyzed. As a result, when the experimenter's primary goal is to complete the test as soon as the total number of failures or the threshold period is recorded, the numerical results reveal that the recommended strategy is adaptable and very helpful in completing the study.</p></abstract>

Posterior Manifolds over Prior Parameter Regions: Beyond Pointwise Sensitivity Assessments for Posterior Statistics from MCMC Inference

Abstract Increases in the use of Bayesian inference in applied analysis, the complexity of estimated models, and the popularity of efficient Markov chain Monte Carlo (MCMC) inference under conjugate priors have led to more scrutiny regarding the specification of the parameters in prior distributions. Impact of prior parameter assumptions on posterior statistics is commonly investigated in terms of local or pointwise assessments, in the form of derivatives or more often multiple evaluations under a set of alternative prior parameter specifications. This paper expands upon these localized strategies and introduces a new approach based on the graph of posterior statistics over prior parameter regions (sensitivity manifolds) that offers additional measures and graphical assessments of prior parameter dependence. Estimation is based on multiple point evaluations with Gaussian processes, with efficient selection of evaluation points via active learning, and is further complemented with derivative information. The application introduces a strategy to assess prior parameter dependence in a multivariate demand model with a high dimensional prior parameter space, where complex prior-posterior dependence arises from model parameter constraints. The new measures uncover a considerable prior dependence beyond parameters suggested by theory, and reveal novel interactions between the prior parameters and the elasticities.

On classical and Bayesian inference for bivariate Poisson conditionals distributions: theory, methods and applications

Bivariate count data arise in several different disciplines (epidemiology, marketing, sports statistics, etc., to name but a few) and the bivariate Poisson distribution which is a generalization of the Poisson distribution plays an important role in modeling such data. In this article, we consider the inferential aspect of a bivariate Poisson conditionals distribution for which both the conditionals are Poisson but the marginals are typically non-Poisson. It has Poisson marginals only in the case of independence. It appears that a simple iterative procedure under the maximum likelihood method performs quite well as compared with other numerical subroutines, as one would expect in such a case where the MLEs are not available in closed form. In the Bayesian paradigm, both conjugate priors and non-conjugate priors have been utilized and a comparison study has been made via a simulation study. For illustrative purposes, a real-life data set is re-analyzed to exhibit the utility of the proposed two methods of estimation, one under the frequentist approach and the other under the Bayesian paradigm.

Posterior marginalization accelerates Bayesian inference for dynamical models of biological processes

Bayesian inference is an important method in the life and natural sciences for learning from data. It provides information about parameter and prediction uncertainties. Yet, generating representative samples from the posterior distribution is often computationally challenging. Here, we present an approach that lowers the computational complexity of sample generation for dynamical models with scaling, offset, and noise parameters. The proposed method is based on the marginalization of the posterior distribution. We provide analytical results for a broad class of problems with conjugate priors and show that the method is suitable for a large number of applications. Subsequently, we demonstrate the benefit of the approach for applications from the field of systems biology. We report an improvement up to 50 times in the effective sample size per unit of time. As the scheme is broadly applicable, it will facilitate Bayesian inference in different research fields.

Hierarchical Bayesian calibration of regional moment magnitude predictive models

Onsite Earthquake Early Warning (EEW) applications require robust predictive models that can relate the early part of the recorded seismic waves to seismicity and/or ground shaking characteristics, the latter representing the variables used to make decisions about raising or not an alarm. The use of limited information to establish these models within a real-time estimation setting, typically corresponding to the first few seconds after the identification of the seismic wave arrival, introduces significant uncertainty (variability) in the predictions. To increase the number of historical observations available for the model calibration, data from multiple seismic stations within the same geographic region are utilized for the latter task. Unfortunately, this introduces additional sources of variability in the predictive model development, originating from local geographical effects and site conditions that affect differently the underlying physics at each station. This paper introduces a hierarchical Bayesian model updating framework to address this challenge. To better focus the research effort, the calibration of earthquake magnitude nonlinear regression models based on the seismic maximum predominant period is investigated, though underlying principles can be extended to other type of decision variables and intensity measures used in EEW applications. Within the hierarchical updating framework, the predictive models for each station are separately calibrated (first level), while sharing information across the stations by selecting common calibration hyper-parameters (second level). This facilitates a reduction of the variability of the estimates originating from regional wave propagation effects while simultaneously providing sufficient amount of data for calibrating the predictive models even for stations with scarce available records. To support computational efficiency in the hierarchical calibration, conjugate priors are utilized, while for accommodating nonlinear regression relationships samples from the posterior distribution, representing the model calibration, are obtained using Metropolis-within-Gibbs sampling. As illustrative case study, the application to the Sichuan region of Southwestern China is examined, with extensive comparisons performed for the accuracy and robustness of the resultant probabilistic predictive models.