Bayesian Way Research Articles

The Bayesian Group-Sequential Predictive Evidence Value Design for Phase II Clinical Trials with Binary Endpoints

AbstractIn clinical research, the initial efficacy of a new agent is typically assessed in a phase IIA study. Bayesian group-sequential designs are often based on predictive probability of trial success. In this paper, the novel Bayesian group-sequential predictive evidence value design is introduced, and we prove that the predictive probability approach is a special case of it. A comparison with Simon’s two-stage and competing Bayesian designs based on phase IIA cancer trials is provided. Results show that the novel design can improve operating characteristics such as the false-positive rate, probability of early stopping for futility and expected sample size of the trial. Given these advantages, the predictive evidence value design constitutes an important addition to the biostatistician’s toolbelt when planning a phase IIA trial the Bayesian way, in particular, when small sample sizes and a large probability for early termination under the null hypothesis are desired.

AovBay: An R Package for Application and Visualization of Parametric Non-parametric and Bayesian ANOVA

The analysis of variance is a statistical technique widely used in the design of experiments and different research areas. It has been modeled using a classical or frequentist approach. With the computational power that is currently available, the Bayesian approach is an essential statistical tool related to hypothesis testing. However, conformity with classical techniques, ignorance of Bayesian statistics, and lack of easy-to-use software are obstacles to its frequent application. In this work, the use of a reproducible statistical package in R is proposed. It presents options to perform an analysis of variance in a classical (frequentist) and Bayesian way when the assumptions of the frequentist approach are not met or when a level of more specific inference such as quantifying the evidence provided by a data set for a given hypothesis, with the possibility of contributing to the understanding of the rejection or not of the statistical hypotheses raised, through interactive graphics presented in an emerging Shiny panel.

A Bayesian method to mitigate the effects of unmodelled time-varying systematics for 21-cm cosmology experiments

ABSTRACT Radio observations of the neutral hydrogen signal from the Cosmic Dawn and Epoch of Reionization have helped to provide constraints on the properties of the first stars and galaxies. Since this global 21-cm cosmological signal from the Cosmic Dawn is effectively constant on observing time-scales and since effects resulting from systematics will vary with time, the effects of these systematics can be mitigated without the need for a model of the systematic. We present a method to account for unmodelled time-varying systematics in 21-cm radio cosmology experiments using a squared exponential Gaussian process kernel to account for correlations between time bins in a fully Bayesian way. We find by varying the model parameters of a simulated systematic that the Gaussian process method improves our ability to recover the signal parameters by widening the posterior in the presence of a systematic and reducing the bias in the mean fit parameters. When varying the amplitude of a model sinusoidal systematic between 0.25 and 2.00 times the 21-cm signal amplitude and the period between 0.5 and 4.0 times the signal width, we find on average a 5 per cent improvement in the root mean squared error of the fitted signal. We can use the fitted Gaussian process hyperparameters to identify the presence of a systematic in the data, demonstrating the method’s utility as a diagnostic tool. Furthermore, we can use Gaussian process regression to calculate a mean fit to the residuals over time, providing a basis for producing a model of the time-varying systematic.

Bayesian learners in gradient boosting for linear mixed models.

Selection of relevant fixed and random effects without prior choices made from possibly insufficient theory is important in mixed models. Inference with current boosting techniques suffers from biased estimates of random effects and the inflexibility of random effects selection. This paper proposes a new inference method "BayesBoost" that integrates a Bayesian learner into gradient boosting with simultaneous estimation and selection of fixed and random effects in linear mixed models. The method introduces a novel selection strategy for random effects, which allows for computationally fast selection of random slopes even in high-dimensional data structures. Additionally, the new method not only overcomes the shortcomings of Bayesian inference in giving precise and unambiguous guidelines for the selection of covariates by benefiting from boosting techniques, but also provides Bayesian ways to construct estimators for the precision of parameters such as variance components or credible intervals, which are not available in conventional boosting frameworks. The effectiveness of the new approach can be observed via simulation and in a real-world application.

Reconstructing AGN X-ray spectral parameter distributions with Bayesian methods

We present a new Bayesian method for reconstructing the parent distributions of X-ray spectral parameters of active galactic nuclei (AGN) in large surveys. The method uses the probability distribution function (PDF) of posteriors obtained by fitting a consistent physical model to each object with a Bayesian method. The PDFs are often broadly distributed and may present systematic biases, such that naive point estimators or even some standard parametric modeling are not sufficient to reconstruct the parent population without obvious bias. Our method uses a transfer function computed from a large realistic simulation with the same selection as in the actual sample to redistribute the stacked PDF and then forward-fit a nonparametric model to it in a Bayesian way, so that the biases in the PDFs are properly taken into account. In this way, we are able to accurately reconstruct the parent distributions. We apply our spectral fitting and population inference methods to the XMM-COSMOS survey as a pilot study. For the 819 AGN detected in the COSMOS field, 663 (81%) of which have spectroscopic redshifts (spec-z) and the others high-quality photometric redshifts (photo-z), we find prominent bi-modality with widely separated peaks in the distribution of the absorbing hydrogen column density (NH) and an indication that absorbed AGN have harder photon indices. A clear decreasing trend of the absorbed AGN fraction versus the intrinsic 2–10 keV luminosity is observed, but there is no clear evolution in the absorbed fraction with redshift. Our method is designed to be readily applicable to large AGN samples such as the XXL survey, and eventually eROSITA.

Clustering consistency with Dirichlet process mixtures

Summary Dirichlet process mixtures are flexible nonparametric models, particularly suited to density estimation and probabilistic clustering. In this work we study the posterior distribution induced by Dirichlet process mixtures as the sample size increases, and more specifically focus on consistency for the unknown number of clusters when the observed data are generated from a finite mixture. Crucially, we consider the situation where a prior is placed on the concentration parameter of the underlying Dirichlet process. Previous findings in the literature suggest that Dirichlet process mixtures are typically not consistent for the number of clusters if the concentration parameter is held fixed and data come from a finite mixture. Here we show that consistency for the number of clusters can be achieved if the concentration parameter is adapted in a fully Bayesian way, as commonly done in practice. Our results are derived for data coming from a class of finite mixtures, with mild assumptions on the prior for the concentration parameter and for a variety of choices of likelihood kernels for the mixture.

Transfer learning of degradation modeling and prognosis based on multivariate functional analysis with heterogeneous sampling rates

Multivariate functional principal component analysis (MFPCA) is a widely used tool for modeling and prognosis of degradation signals. However, MFPCA usually assumes samples from each process share the same sampling rate. To relax this assumption and facilitate the information sharing among multiple degradation processes, we propose a transfer learning framework to conduct modeling and prognosis of degradation data with heterogeneous sampling rates. This framework features the flexible modeling capability and focuses on transferring degradation information from correlated processes to the prognosis of an interested (target) process. More specifically, we use kernel smoothing to handle the heterogeneous sampling rates so that the mean and covariance in each sample can be successfully estimated. Then, the conditional expectation is developed to estimate the MFPC scores, which contain correlation information among processes. These MFPC scores are then updated in a Bayesian way by incorporating the observed data from the in-service unit in the target process. The updated MFPC scores deliver a superior performance of prognosis. The proposed method is validated and compared with various benchmarks in extensive numerical studies and one case study. The results show the proposed method can successfully handle the issue of heterogeneous sampling rates and provide an improved prognosis result.

Optimising inflationary features the Bayesian way

Modern cosmological data demand modern data analysis techniques. We introduce BayOp, a new likelihood sampling and maximisation method which is based on the Bayesian Optimisation algorithm and learns a function instead of randomly sampling from it. We apply BayOp to analyse Planck data for traces of inflationary features models with global periodic modulations of the primordial power spectrum. While we do not find any new evidence for features, we demonstrate that BayOp provides an extremely efficient way of sampling likelihoods over low-to-moderate-dimensional parameter spaces, even for very complex likelihood landscapes.

Block-Term Tensor Decomposition Model Selection and Computation: The Bayesian Way

The so-called block-term decomposition (BTD) tensor model, especially in its rank-$(L_r,L_r,1)$ version, has been recently receiving increasing attention due to its enhanced ability of representing systems and signals that are composed of \emph{blocks} of rank higher than one, a scenario encountered in numerous and diverse applications. Uniqueness conditions and fitting methods have thus been thoroughly studied. Nevertheless, the challenging problem of estimating the BTD model structure, namely the number of block terms, $R$, and their individual ranks, $L_r$, has only recently started to attract significant attention, mainly through regularization-based approaches which entail the need to tune the regularization parameter(s). In this work, we build on ideas of sparse Bayesian learning (SBL) and put forward a fully automated Bayesian approach. Through a suitably crafted multi-level \emph{hierarchical} probabilistic model, which gives rise to heavy-tailed prior distributions for the BTD factors, structured sparsity is \emph{jointly} imposed. Ranks are then estimated from the numbers of blocks ($R$) and columns ($L_r$) of non-negligible energy. Approximate posterior inference is implemented, within the variational inference framework. The resulting iterative algorithm completely avoids hyperparameter tuning, which is a significant defect of regularization-based methods. Alternative probabilistic models are also explored and the connections with their regularization-based counterparts are brought to light with the aid of the associated maximum a-posteriori (MAP) estimators. We report simulation results with both synthetic and real-word data, which demonstrate the merits of the proposed method in terms of both rank estimation and model fitting as compared to state-of-the-art relevant methods.

Towards another paradigm for forensic science?

AbstractDaubert skews the contribution of forensic science because it only took into account its Galilean dimension (construction of general predictive models). However, forensic science should better be classified in the historical sciences (clinical approach to reconstruct a past event of presence or activity). We therefore need a complementary approach that integrates the necessarily “clinical” part in the resolution of forensic issues. Such an evolution involves semiotics. While recognizing that the Bayesian way of thinking is the only prescriptive available model for interpretation fitting well in the Galilean paradigm, the complexity of the reconstruction of a past‐uncontrolled singular case and the robustness of available relevant data to it, invites consideration of its implementation in a semiotic line of arguments. Indeed, Bayes makes it possible to remain in a single harmonized model integrating both the clinical and Galilean dimensions, but rapidly the complexity of the modeling and its mathematization come up against more qualitative natural and legal reasoning. Two different systems of reasoning at stake are inevitably creating a “bug” that could explain the current forensic crisis and miscarriages of justice. This anomaly is reflected in the issue of transparency (misunderstandings by and between interlocutors on the nature of the expertise, if not science). Peirce offers a path to address the tension between complementary reasoning systems.This article is categorized under:Crime Scene Investigation > Epistemology and MethodCrime Scene Investigation > From Traces to Intelligence and EvidenceCrime Scene Investigation > Education and Formation

Testing temporal integration of feature probability distributions using role-reversal effects in visual search

The visual system is sensitive to statistical properties of complex scenes and can encode feature probability distributions in detail. But does the brain use these statistics to build probabilistic models of the ever-changing visual input? To investigate this, we examined how observers temporally integrate two different orientation distributions from sequentially presented visual search trials. If the encoded probabilistic information is used in a Bayesian optimal way, observers should weigh more reliable information more strongly, such as feature distributions with low variance. We therefore manipulated the variance of the two feature distributions. Participants performed sequential odd-one-out visual search for an oddly oriented line among distractors. During successive learning trials, the distractor orientations were sampled from two different Gaussian distributions on alternating trials. Then, observers performed a ‘test trial’ where the orientations of the target and distractors were switched, allowing us to assess observer’s internal representation of distractor distributions based on changes in response times. In three experiments we observed that observer’s search times on test trials depended mainly on the very last learning trial, indicating a strong recency effect. Since temporal integration has been previously observed with this method, we conclude that when the input is unreliable, the visual system relies more on the most recent stimulus. This indicates that the visual system prefers to utilize sensory history when the statistical properties of the environment are relatively stable.

Constrains on the electric charges of the binary black holes with GWTC-1 events

Testing black hole’s charged property is a fascinating topic in modified gravity and black hole astrophysics. In the first Gravitational-Wave Transient Catalog (GWTC-1), ten binary black hole merger events have been formally reported, and these gravitational wave signals have significantly enhanced our understanding of the black hole. In this paper, we try to constrain the amount of electric charge with the parameterized post-Einsteinian framework by treating the electric charge as a small perturbation in a Bayesian way. We find that the current limits in our work are consistent with the result of Fisher information matrix method in previous works. We also develop a waveform model considering a leading order charge effect for binary black hole inspiral.

Win-Stay-Lose-Shift as a self-confirming equilibrium in the iterated Prisoner's Dilemma.

Evolutionary game theory assumes that players replicate a highly scored player’s strategy through genetic inheritance. However, when learning occurs culturally, it is often difficult to recognize someone’s strategy just by observing the behaviour. In this work, we consider players with memory-one stochastic strategies in the iterated Prisoner’s Dilemma, with an assumption that they cannot directly access each other’s strategy but only observe the actual moves for a certain number of rounds. Based on the observation, the observer has to infer the resident strategy in a Bayesian way and chooses his or her own strategy accordingly. By examining the best-response relations, we argue that players can escape from full defection into a cooperative equilibrium supported by Win-Stay-Lose-Shift in a self-confirming manner, provided that the cost of cooperation is low and the observational learning supplies sufficiently large uncertainty.

The impact of cognitive and noncognitive factors on odor discrimination performance

AbstractThis study aimed to investigate which components among a range of cognitive and noncognitive factors contribute to odor discrimination and to which extent. One hundred sixty participants were assessed for olfactory discrimination considering (a) the sequence of presentation of the detection target and (b) response time taken to perform the discrimination task. Participants were tested with several cognitive tests covering executive functioning, and semantic memory and noncognitive tests measuring optimism, mindset, general self‐efficacy, and self‐esteem. Hierarchical regression analyses conducted in classic and Bayesian ways demonstrate a strong impact of semantic memory and age on odor discrimination performance. Conversely, executive functions together with noncognitive factors and response time were rather weak predictors of odor discrimination performance. Additionally, the positioning of the target odor, when presented last, facilitated the correct choice in odor discrimination. In conclusion, odor discrimination is related to semantic memory, while noncognitive factors exert only minor influences on odor discrimination scores.Practical ApplicationsIn the present study, we found that the target odor, when presented as the last of the three pens, facilitated the correct choice in odor discrimination process. Even though, the influence of the target odor position was noticed only in case of several trials, it should be considered in the future studies. Moreover, the amount of time taken to respond when performing odor discrimination task did not contribute strongly to the final outcome. Thus, controlling for the response time in this kind of task appears not to be highly significant.

Bayesian Shape Invariant Model for Latent Growth Curve with Time-Invariant Covariates

In the attention-deficit hyperactivity disorder (ADHD) study, children are prescribed different stimulant medications. The height measurements are recorded longitudinally along with the medication time. Differences among the patients are captured by the parameters suggested the Superimposition by Translation and Rotation (SITAR) model using three subject-specific parameters to estimate their deviation from the mean growth curve. In this paper, we generalize the SITAR model in a Bayesian way with time-invariant covariates. The time-invariant model allows us to predict latent growth factors. Since patients suffer from a common disease, they usually exhibit a similar pattern, and it is natural to build a nonlinear model that is shaped invariant. The model is semi-parametric, where the population time curve is modeled with a natural cubic spline. The original shape invariant growth curve model, motivated by epidemiological research on the evolution of pubertal heights over time, fits the underlying shape function for height over age and estimates subject-specific deviations from this curve in terms of size, tempo, and velocity using maximum likelihood. The usefulness of the model is illustrated in the attention deficit hyperactivity disorder (ADHD) study. Further, we demonstrated the effect of stimulant medications on pubertal growth by gender.

A mixed hidden Markov model for multivariate monotone disease processes in the presence of measurement errors

Motivated by a longitudinal oral health study, the Signal-Tandmobiel® study, an inhomogeneous mixed hidden Markov model with continuous state-space is proposed to explain the caries disease process in children between 6 and 12 years of age. The binary caries experience outcomes are subject to misclassification. We modelled this misclassification process via a longitudinal latent continuous response subject to a measurement error process and showing a monotone behaviour. The baseline distributions of the unobservable continuous processes are defined as a function of the covariates through the specification of conditional distributions making use of the Markov property. In addition, random effects are considered to model the relationships among the multivariate responses. Our approach is in contrast with a previous approach working on the binary outcome scale. This method requires conditional independence of the possibly corrupted binary outcomes on the true binary outcomes. We assumed conditional independence on the latent scale, which is a weaker assumption than conditional independence on the binary scale. The aim of this article is therefore to show the properties of a model for a progressive longitudinal response with misclassification on the manifest scale but modelled on the latent scale. The model parameters are estimated in a Bayesian way using an efficient Markov chain Monte Carlo method. The model performance is shown through a simulation-based example, and the analysis of the motivating dataset is presented.

Measuring the Uncertainty of Predictions in Deep Neural Networks with Variational Inference.

We present a novel approach for training deep neural networks in a Bayesian way. Compared to other Bayesian deep learning formulations, our approach allows for quantifying the uncertainty in model parameters while only adding very few additional parameters to be optimized. The proposed approach uses variational inference to approximate the intractable a posteriori distribution on basis of a normal prior. By representing the a posteriori uncertainty of the network parameters per network layer and depending on the estimated parameter expectation values, only very few additional parameters need to be optimized compared to a non-Bayesian network. We compare our approach to classical deep learning, Bernoulli dropout and Bayes by Backprop using the MNIST dataset. Compared to classical deep learning, the test error is reduced by 15%. We also show that the uncertainty information obtained can be used to calculate credible intervals for the network prediction and to optimize network architecture for the dataset at hand. To illustrate that our approach also scales to large networks and input vector sizes, we apply it to the GoogLeNet architecture on a custom dataset, achieving an average accuracy of 0.92. Using 95% credible intervals, all but one wrong classification result can be detected.

Evidence of Absence: Bayesian Way to Reveal True Zeros Among Occupational Exposures.

Workplace exposure measurements typically contain some observations below limit of detection. The current paradigm for exposure data interpretation relies on the lognormal distribution, where censored observation are assumed to be present but not quantifiable. However, there are setting were such assumptions are untenable and true zero exposures cannot be ruled out. This issue can be non-trivial because decisions about compliance depend on the adequacy of the lognormal model. We adapted previously described statistical models for mixture of true zeros and lognormal distribution to function within Bayesian procedure that overcomes historical limitations that precluded them from being used in practice. We compared the performance of the new models and the traditional lognormal model in simulation. Their implementation is illustrated in diverse datasets. The approach we propose involves estimating the proportion of true zeroes, and the geometric mean and standard deviation of the lognormal component of the mixture. This can be implemented in practice either based on the truncated lognormal model fit to the observed data, or on the censored Bernoulli-lognormal mixture model, which has the advantage of allowing for multiple censoring points. Both models can be implemented via a free online application. In simulations, when none of the censored values were zeros, all estimation procedures led to similar risk assessment. However, when all or most of the censored values were zeros, the traditional approach that assumes lognormal distribution performed noticeably worse than newly proposed methods, typically overestimating noncompliance. Application to real data suggests that we cannot rule out presence of true zero exposures in typical measurement series gathered by occupational hygienists. Forcing the usual lognormal model to data containing a large proportion of censored values can bias risk assessment if a substantial part of the censored points are true zeroes. The Bernoulli-lognormal model is a suitable and accessible model that can account for such challenging data, and leads to unbiased risk assessments regardless of the presence of true zeros in the data.

Estimation of COVID-19 spread curves integrating global data and borrowing information.

Currently, novel coronavirus disease 2019 (COVID-19) is a big threat to global health. The rapid spread of the virus has created pandemic, and countries all over the world are struggling with a surge in COVID-19 infected cases. There are no drugs or other therapeutics approved by the US Food and Drug Administration to prevent or treat COVID-19: information on the disease is very limited and scattered even if it exists. This motivates the use of data integration, combining data from diverse sources and eliciting useful information with a unified view of them. In this paper, we propose a Bayesian hierarchical model that integrates global data for real-time prediction of infection trajectory for multiple countries. Because the proposed model takes advantage of borrowing information across multiple countries, it outperforms an existing individual country-based model. As fully Bayesian way has been adopted, the model provides a powerful predictive tool endowed with uncertainty quantification. Additionally, a joint variable selection technique has been integrated into the proposed modeling scheme, which aimed to identify possible country-level risk factors for severe disease due to COVID-19.

Scientific self-correction: the Bayesian way

The enduring replication crisis in many scientific disciplines casts doubt on the ability of science to estimate effect sizes accurately, and in a wider sense, to self-correct its findings and to produce reliable knowledge. We investigate the merits of a particular countermeasure—replacing null hypothesis significance testing (NHST) with Bayesian inference—in the context of the meta-analytic aggregation of effect sizes. In particular, we elaborate on the advantages of this Bayesian reform proposal under conditions of publication bias and other methodological imperfections that are typical of experimental research in the behavioral sciences. Moving to Bayesian statistics would not solve the replication crisis single-handedly. However, the move would eliminate important sources of effect size overestimation for the conditions we study.