Bayesian Hypothesis Testing Research Articles

Null results of oxytocin and vasopressin administration across a range of social cognitive and behavioral paradigms: Evidence from a randomized controlled trial

Research examining oxytocin and vasopressin in humans has the potential to elucidate neurobiological mechanisms underlying human sociality that have been previously unknown or not well characterized. A primary goal of this work is to increase our knowledge about neurodevelopmental and psychiatric disorders characterized by impairments in social cognition. However, years of research highlighting wide-ranging effects of, in particular, intranasal oxytocin administration have been tempered as the fields of psychology, neuroscience, and other disciplines have been addressing concerns over the reproducibility and validity of research findings. We present a series of behavioral tasks that were conducted using a randomized, double-blind, placebo controlled, between-subjects design, in which our research group found no main effects of oxytocin and vasopressin on a host of social outcomes. In addition to null hypothesis significance testing, we implemented equivalence testing and Bayesian hypothesis testing to examine the sensitivity of our findings. These analyses indicated that 47–83% of our results (depending on the method of post-hoc analysis) had enough sensitivity to detect the absence of a main effect. Our results add to evidence that intranasal oxytocin may have a more limited direct effect on human social processes than initially assumed and suggest that the direct effects of intranasal vasopressin may be similarly limited. Randomized controlled trial registration: NCT01680718.

A tutorial on generalizing the default Bayesian t-test via posterior sampling and encompassing priors

With the advent of so-called default Bayesian hypothesis tests, scientists in applied fields have gained access to a powerful and principled method for testing hypotheses. However, such default tests usually come with a compromise, requiring the analyst to accept a one-size-fits-all approach to hypothesis testing. Further, such tests may not have the flexibility to test problems the scientist really cares about. In this tutorial, I demonstrate a flexible approach to generalizing one specific default test (the JZS t-test; Rouder et al., 2009) that is becoming increasingly popular in the social and behavioral sciences. The approach uses two theoretical results, the Savage-Dickey density ratio (Dickey and Lientz, 1980) and the technique of encompassing priors (Klugkist et al., 2005) in combination with MCMC sampling via an easy-to-use probabilistic modeling package for R called Greta. Through a comprehensive mathematical description of the techniques as well as illustrative examples, the reader is presented with a general, flexible workflow that can be extended to solve problems relevant to his or her own work.

An overview of Bayesian reasoning in the analysis of delay-discounting data.

Statistical inference (including interval estimation and model selection) is increasingly used in the analysis of behavioral data. As with many other fields, statistical approaches for these analyses traditionally use classical (i.e., frequentist) methods. Interpreting classical intervals and p-values correctly can be burdensome and counterintuitive. By contrast, Bayesian methods treat data, parameters, and hypotheses as random quantities and use rules of conditional probability to produce direct probabilistic statements about models and parameters given observed study data. In this work, we reanalyze two data sets using Bayesian procedures. We precede the analyses with an overview of the Bayesian paradigm. The first study reanalyzes data from a recent study of controls, heavy smokers, and individuals with alcohol and/or cocaine substance use disorder, and focuses on Bayesian hypothesis testing for covariates and interval estimation for discounting rates among various substance use disorder profiles. The second example analyzes hypothetical environmental delay-discounting data. This example focuses on using historical data to establish prior distributions for parameters while allowing subjective expert opinion to govern the prior distribution on model preference. We review the subjective nature of specifying Bayesian prior distributions but also review established methods to standardize the generation of priors and remove subjective influence while still taking advantage of the interpretive advantages of Bayesian analyses. We present the Bayesian approach as an alternative paradigm for statistical inference and discuss its strengths and weaknesses.

How to (better) find a perpetrator in a haystack

In many real-world contexts, there is a pressing need to automatically screen for potential perpetrators, such as school shooters, whose prevalence in the population is extremely low. We first explain one possible obstacle in addressing this challenge, which is the confusion between “recognition” and “localization” during a search process. Next, we present a pragmatic screening methodology to the problem along Jaynes Bayesian hypothesis testing procedure. According to this approach, we should first focus our efforts on reducing the size of the haystack rather than on the identification of the needle. The third and major methodological contribution of the paper is in proposing that we may reduce the size of the haystack through the identification and use of unique data cues we describe as “impostors’ cues”. An experiment performed on an artificial data set of 7000 texts, shows that when incorporating these cues in the hypothesis testing procedure, they significantly improve the automatic screening of objects characterized by an attribute of a low prevalence (i.e. a psychopathic signature). The relevance of the proposed approach for Big Data and Homeland security is explained and discussed.

A Bayesian Approach to Measuring Evidence in L2 Research: An Empirical Investigation

AbstractNull hypothesis testing has long since been the ‘go‐to analytic approach’ in quantitative second language (L2) research (Norris, 2015, p. 97). To many, however, years of reliance on this approach has resulted in a crisis of inference across the social and behavioral sciences (e.g., Rouder et al., 2016). As an alternative to the null hypothesis testing approach, many such experts recommend the Bayesian hypothesis testing approach. Adopting an open‐science framework, the present study (a) re‐evaluates the empirical findings of 418 t‐tests from published L2 research using Bayesian hypothesis testing, and (b) compares the Bayesian results with their conventional, null hypothesis testing counterparts as observed in the original reports. The results show that the Bayesian and the null hypothesis testing approaches generally arrive at similar inferential conclusions. However, considerable differences arise in the rejections of the null hypothesis. Notably, in 64.06% of cases when p‐values fell between .01 and .05 (i.e., evidence to reject the null), the Bayesian analysis found the evidence in the primary studies to be only at an ‘anecdotal’ level (i.e., insufficient evidence to reject the null). Practical implications, field‐wide recommendations, and an introduction to free online software (https://rnorouzian.shinyapps.io/bayesian-t-tests) for Bayesian hypothesis testing are discussed.

Bayesian fuzzy hypothesis test in wireless sensor networks with noise uncertainty

Reliable event detection is an essential task of wireless sensor networks (WSNs) in which there are different types of uncertainty. In this paper, we consider a decentralized detection problem for a WSN and use fuzzy hypothesis test (FHT) in the Bayesian perspective to model the noise power uncertainty. FHT employs membership functions as hypotheses for modeling and analyzing the uncertainty. Using Bayesian FHT (BFHT), a local detector scheme is proposed at each sensor node in which the threshold depends on the noise power uncertainty bound. Local decisions of sensors are sent to the fusion center (FC) and combined to make a final decision about the absence/presence of the event. The proposed algorithm is evaluated in terms of probabilities of detection and false alarm. Simulations show that the proposed BFHT detector considerably outperforms the Anderson–Darling method as well as the conventional energy detector in the presence of the noise power uncertainty.

On Bayesian Analysis and Hypothesis Testing in the Determination of Bioequivalence.

On Bayesian Analysis and Hypothesis Testing in the Determination of Bioequivalence.

JASP: Graphical Statistical Software for Common Statistical Designs

This paper introduces JASP, a free graphical software package for basic statistical procedures such as t tests, ANOVAs, linear regression models, and analyses of contingency tables. JASP is open-source and differentiates itself from existing open-source solutions in two ways. First, JASP provides several innovations in user interface design; specifically, results are provided immediately as the user makes changes to options, output is attractive, minimalist, and designed around the principle of progressive disclosure, and analyses can be peer reviewed without requiring a “syntax”. Second, JASP provides some of the recent developments in Bayesian hypothesis testing and Bayesian parameter estimation. The ease with which these relatively complex Bayesian techniques are available in JASP encourages their broader adoption and furthers a more inclusive statistical reporting practice. The JASP analyses are implemented in R and a series of R packages. © 2019, American Statistical Association. All rights reserved.

Bayesian inference for the common location parameter of several shifted-exponential populations

Consider k random samples which are independently drawn from k shifted-exponential distributions, with respective scale parameters σ 1 , σ 2 , … , σ k and common location parameter θ . On the basis of the given samples and in a Bayesian framework, we address the problem of point and interval estimation of the location parameter θ under the conjugate priors, which are usually proper priors. Moreover, we also address the problem of testing the equality of the location parameters. We propose Bayesian hypothesis testing procedures for the equality of the location parameters under the noninformative prior. The noninformative prior is usually improper which yields a calibration problem that makes the Bayes factor to be defined up to a multiplicative constant. So we propose the default Bayesian hypothesis testing procedures based on the fractional Bayes factor and the intrinsic Bayes factors under the reference priors. Our proposed Bayesian procedures are compared and contrasted, via a comparison study, a simulation study, and a real-world data analysis, to the existing classical exact procedures proposed by Tippett (Tippett’s method (Tippett, 1931)), Fisher (Fisher’s method (Fisher, 1932)), Stouffer (Inverse normal method (Stouffer et al., 1949)), and George (Logit method (George, 1977)), and to the generalized variable procedures proposed by Tsui and Weerahandi (Generalized p -value method (Tsui and Weerahandi, 1989)).

Multiclass classification, information, divergence and surrogate risk

We provide a unifying view of statistical information measures, multiway Bayesian hypothesis testing, loss functions for multiclass classification problems and multidistribution $f$-divergences, elaborating equivalence results between all of these objects, and extending existing results for binary outcome spaces to more general ones. We consider a generalization of $f$-divergences to multiple distributions, and we provide a constructive equivalence between divergences, statistical information (in the sense of DeGroot) and losses for multiclass classification. A major application of our results is in multiclass classification problems in which we must both infer a discriminant function $\gamma$—for making predictions on a label $Y$ from datum $X$—and a data representation (or, in the setting of a hypothesis testing problem, an experimental design), represented as a quantizer $\mathsf{q}$ from a family of possible quantizers $\mathsf{Q}$. In this setting, we characterize the equivalence between loss functions, meaning that optimizing either of two losses yields an optimal discriminant and quantizer $\mathsf{q}$, complementing and extending earlier results of Nguyen et al. [Ann. Statist. 37 (2009) 876–904] to the multiclass case. Our results provide a more substantial basis than standard classification calibration results for comparing different losses: we describe the convex losses that are consistent for jointly choosing a data representation and minimizing the (weighted) probability of error in multiclass classification problems.

On the Optimality of Likelihood Ratio Test for Prospect Theory-Based Binary Hypothesis Testing

In this letter, the optimality of the likelihood ratio test (LRT) is investigated for binary hypothesis testing problems in the presence of a behavioral decision-maker. By utilizing prospect theory, a behavioral decision-maker is modeled to cognitively distort probabilities and costs based on some weight and value functions, respectively. It is proved that the LRT may or may not be an optimal decision rule for prospect theory based binary hypothesis testing and conditions are derived to specify different scenarios. In addition, it is shown that when the LRT is an optimal decision rule, it corresponds to a randomized decision rule in some cases; i.e., nonrandomized LRTs may not be optimal. This is unlike Bayesian binary hypothesis testing in which the optimal decision rule can always be expressed in the form of a nonrandomized LRT. Finally, it is proved that the optimal decision rule for prospect theory based binary hypothesis testing can always be represented by a decision rule that randomizes at most two LRTs. Two examples are presented to corroborate the theoretical results.

Bayesian Monte Carlo testing with one-dimensional measures of evidence

Bayesian hypothesis testing procedures are constructed by means of test statistics which are functions of the posterior distribution. Usually, the whole sample vector is selected to form the sufficient empirical part of the posterior distribution. But, in certain problems, one may prefer to use well-established one-dimensional sufficient statistics in place of the sample vector. This paper introduces a Bayesian Monte Carlo procedure specially designed for such cases. It is shown that the performance of this new approach is arbitrarily close to the exact Bayesian test. In addition, for arbitrary desired precisions, we develop a theoretical rule of thumb for choosing the minimum number m 0 of Monte Carlo simulations. Surprisingly, m 0 does not depend on the shape of loss/cost functions when those are used to compound the test statistic. The method is illustrated for testing mean vectors in high-dimension and for detecting spatial clusters of diseases in aggregated maps.

Serotonin 5-HT1A receptor binding and self-transcendence in healthy control subjects—a replication study using Bayesian hypothesis testing

ObjectiveA putative relationship between markers for the serotonin system and the personality scale self-transcendence (ST) and its subscale spiritual acceptance (SA) has been demonstrated in a previous PET study of 5-HT1A receptor binding in healthy control subjects. The results could however not be replicated in a subsequent PET study at an independent centre. In this study, we performed a replication of our original study in a larger sample using Bayesian hypothesis testing to evaluate relative evidence both for and against this hypothesis.MethodsRegional 5-HT1A receptor binding potential (BPND) was examined in 50 healthy male subjects using PET with the radioligand [11C]WAY100635. 5-HT1Aavailability was calculated using the simplified reference tissue model (SRTM) yielding regional BPND. ST and SA were measured using the Temperament and Character Inventory (TCI) questionnaire. Correlations between ST/SA scores and 5-HT1ABPND in frontal cortex, hippocampus and raphe nuclei were examined by calculation of default correlation Bayes factors (BFs) and replication BFs.ResultsThere were no significant correlations between 5-HT1A receptor binding and ST/SA scores. Rather, five of six replication BFs provided moderate to strong evidence for no association between 5-HT1A availability and ST/SA, while the remaining BF provided only weak evidence.ConclusionWe could not replicate our previous findings of an association between 5-HT1A availability and the personality trait ST/SA. Rather, the Bayesian analysis provided evidence for a lack of correlation. Further research should focus on whether other components of the serotonin system may be related to ST or SA. This study also illustrates how Bayesian hypothesis testing allows for greater flexibility and more informative conclusions than traditional p-values, suggesting that this approach may be advantageous for analysis of molecular imaging data.

Using Kernel Density Estimation to Detect Loss-of-Coolant Accidents in a Pressurized Water Reactor

This paper presents data-driven methods to detect loss-of-coolant accidents (LOCAs) in the primary side of a pressurized water reactor. Process data for a variety of accident scenarios have been generated and collected using a generic pressurized water reactor simulator. The data have been used to train kernel density functions, which estimate nonparametric probability density functions based on training data. These density functions have then been used with Bayesian hypothesis testing and maximum likelihood estimation to detect the onset of the LOCAs and to identify where in the primary side the leaks have occurred. The methods have been able to detect the LOCAs for all scenarios tested with an average detection delay of one-seventh the time for the reactor to trip. Furthermore, the methods have been able to correctly identify the leak locations for 92.3% of the scenarios tested, with higher success rates for larger leaks.

Bayesian uncertainty quantification and information fusion in CALPHAD-based thermodynamic modeling

Calculation of phase diagrams is one of the fundamental tools in alloy design—more specifically under the framework of Integrated Computational Materials Engineering. Uncertainty quantification of phase diagrams is the first step required to provide confidence for decision making in property- or performance-based design. As a manner of illustration, a thorough probabilistic assessment of the CALPHAD model parameters is performed against the available data for a Hf–Si binary case study using a Markov Chain Monte Carlo sampling approach. The plausible optimum values and uncertainties of the parameters are thus obtained, which can be propagated to the resulting phase diagram. Using the parameter values obtained from deterministic optimization in a computational thermodynamic assessment tool (in this case Thermo-Calc) as the prior information for the parameter values and ranges in the sampling process is often necessary to achieve a reasonable cost for uncertainty quantification. This brings up the problem of finding an appropriate CALPHAD model with high-level of confidence which is a very hard and costly task that requires considerable expert skill. A Bayesian hypothesis testing based on Bayes’ factors is proposed to fulfill the need of model selection in this case, which is applied to compare four recommended models for the Hf–Si system. However, it is demonstrated that information fusion approaches, i.e., Bayesian model averaging and an error correlation-based model fusion, can be used to combine the useful information existing in all the given models rather than just using the best selected model, which may lack some information about the system being modelled.

Bayesian Modeling of Working Memory and Inhibitory Control

In cognitive science, working memory is a core cognitive ability that might be functionally related to other capacities, such as perceptual processes, inhibitory control, memory and attention processes and executive functions. The mathematical study of working memory has been explored before. However, there is not enough research aiming to study the relationship between working memory and inhibitory control. This is the objective of the present report. Bayesian hypothesis testing is often more robust than traditional p-value null hypothesis testing. Yet, the number of studies using this approach is still limited. A secondary objective of this paper is to contribute to fill that gap, as well as provide an empirical application of Bayesian hypothesis testing using cognitive and behavioral data. A within-subjects design was used to measure working memory function for three types of visual stimuli that varied in the degree of attentional interference they were designed to elicit. Data collected was contrasted with measurements of inhibitory control and analyzed using Bayes’ theorem. Our results provide evidence against the theoretical relationship of working memory and inhibitory control. This outcome is analyzed in light of related cognitive research.

New perspectives on binding in visual working memory.

How does visual working memory (WM) store the binding between different features of a visual object (like colour, orientation, and location), and does memorizing these bindings require additional resources beyond memorizing individual features? These questions have traditionally been addressed by comparing performance across different types of change detection task. More recently, experimental tasks such as analogue (cued) recall, combined with analysis methods including Bayesian hypothesis testing and formal model comparison, have shed new light on the properties of WM. A significant new perspective is that noise in neural representation limits the precision of recall, and several recent models incorporate this view to account for failures of binding in WM. We review the literature on feature binding with a focus on these new developments and discuss their implications for the interpretation of classical findings.

Legal Sufficiency of Statistical Evidence

When are litigants' statistical estimates legally sufficient, given that courts use the preponderance of the evidence standard? We answer this question using Bayesian hypothesis testing and principles of federal procedural law, focusing on the common case of statistical estimation evidence from a normally distributed estimator. Our core result is that mathematical statistics and black-letter law combine to create a simple standard: statistical estimation evidence is legally sufficient when it fits the litigation position of the party relying on it. This means statistical estimation evidence is legally sufficient when the p-value is less than 0.5; equivalently, the preponderance standard is frequentist hypothesis testing with a significance level of just below 0.5. Finally, we show that conventional significance levels such as 0.05 require elevated standards of proof tantamount to clear-and-convincing or beyond-a-reasonable-doubt.

Revisiting the Paleomagnetic Reversal Test: A Bayesian Hypothesis Testing Framework for a Common Mean Direction

AbstractFor over 60 years a suite of field tests has provided paleomagnetists with a means to evaluate the timing of magnetic remanence acquisition and subsequent remanence stability. These tests are crucial for detecting potential overprinting by secondary remanences and for assessing the fidelity of the geological information carried by paleomagnetic signals. The reversal test was developed to detect secondary remanent magnetizations that could potentially bias paleomagnetic reconstructions. More recently, the reversal test has been applied to a broader set of problems, which require statistical assessment of common or antipodal paleomagnetic directions. From a statistical standpoint, the reversal test must distinguish whether two sets of paleomagnetic directions originate from populations with a common mean. However, earlier work has demonstrated that the reversal test may be ambiguous for small numbers of observations because insufficient information is available to reject the null hypothesis of a common mean direction. Here we develop a Bayesian framework to estimate directly the probability that two Fisher‐distributed sets of directions originate from populations with a common mean. This framework can be used to consider data sets with common or different precisions and, thus, provides a fully probabilistic version of the parametric reversal test. Additionally, adoption of a Bayesian framework means that ambiguity associated with the lack of information provided by small numbers of observations is incorporated into the final probability estimate in a natural way. Our new Bayesian test is demonstrated with numerical examples and case studies.

Attack-Aware Defense Strategy: A Robust Cooperative Spectrum Sensing in Cognitive Radio Sensor Networks

Cognitive radio (CR) is a key enabling technology of dynamic spectrum access to achieve better utilization of frequency spectrum. Cooperative spectrum sensing (CSS) exploits the spatial diversity between CR users to improve the accuracy of spectrum sensing. But, in practice, the reliability of the CSS is vulnerable to spectrum sensing data falsification (SSDF) attack. In this type of attack, some malicious CR sensors intentionally report falsified local sensing results to a fusion center and disrupt the global decision-making process. In this study, we present a smart and robust defense scheme called attack-aware CSS (ACSS). The idea is based on the estimation of attack strength where the attack strength corresponds to the attack population. The attack strength is defined as the ratio of the number of malicious sensors to the total number of sensors which is equal to the probability that a specific sensor is malicious. Assuming the malicious sensor behavior or attack strategy, the proposed approach estimates attack strength and uses the Bayesian hypothesis test to improve the collaborative sensing performance. Computer simulation results illustrate the effectiveness of the proposed ACSS approach against SSDF attacks, especially for CR networks located in hostile environments.