Frequentist Tests Research Articles

Risk of Instrumental Delivery in Maternal Obesity: Estimates With Measures of Effect Size.

Background: Obesity, defined as a body mass index ≥30 kg/m2, is epidemic in the United States and is associated with increased risks of adverse events. Studies have examined the role of maternal obesity on the incidence of instrumental vaginal delivery, but the results are divided. However, these analyses used frequentist tests that risk false discovery. The purpose of this retrospective study was to quantify the association of maternal obesity to the need for instrumental delivery with measures of effect size. Measures of effect size allow assessment of the impact of clinical risk factors on outcomes of interest. Methods: All parturients aged ≥18 years in active labor at our facility from January 2018 to May 2019 were entered into this study. Patient demographics, previously reported comorbidities, and obstetric parameters were collected and analyzed to determine the clinical impact of maternal obesity on the incidence of instrumental delivery. One effect size measure, risk differences, was used to quantify the clinical effect of maternal obesity on the need for instrumental delivery. A generalized linear model was used to standardize the measures of effect size of previously reported comorbidities, including maternal obesity, and to determine their association with the need for instrumental delivery. Results: The incidences of chronic and gestational hypertension, preeclampsia, chronic diabetes, and reactive airway disease were higher in parturients with maternal obesity. Risk differences due to maternal obesity were observed in parturients presenting with shoulder dystocia but not in those who underwent oxytocin induction or in nulliparous parturients. Following regression analysis, maternal obesity did not clinically impact the need for instrumental delivery. Conclusion: These findings suggest that maternal obesity did not have a clinical impact on the need for instrumental delivery.

Beyond 'statistical significance': A nontechnical primer of Bayesian statistics and Bayes factors for health researchers.

Hypothesis testing is integral to health research and is commonly completed through frequentist statistics focused on computing p values. p Values have been long criticized for offering limited information about the relationship of variables and strength of evidence concerning the plausibility, presence and certainty of associations among variables. Bayesian statistics is a potential alternative for inference-making. Despite emerging discussion on Bayesian statistics across various disciplines, the uptake of Bayesian statistics in health research is still limited. To offer a primer on Bayesian statistics and Bayes factors for health researchers to gain preliminary knowledge of its use, application and interpretation in health research. Theoretical and empirical literature on Bayesian statistics and methods were used to develop this methodological primer. Using Bayesian statistics in health research without a careful and complete understanding of its underlying philosophy and differences from frequentist testing, estimation and interpretation methods can result in similar ritualistic use as done for p values. Health researchers should supplement frequentists statistics with Bayesian statistics when analysing research data. The overreliance on p values for clinical decisions making should be avoided. Bayes factors offer a more intuitive measure of assessing the strength of evidence for null and alternative hypothesis.

The Bayes factor, HDI-ROPE, and frequentist equivalence tests can all be reverse engineered-Almost exactly-From one another: Reply to Linde et al. (2021).

Following an extensive simulation study comparing the operating characteristics of three different procedures used for establishing equivalence (the frequentist "TOST," the Bayesian "HDI-ROPE," and the Bayes factor interval null procedure), Linde et al. (2021) conclude with the recommendation that "researchers rely more on the Bayes factor interval null approach for quantifying evidence for equivalence" (p. 1). We redo the simulation study of Linde et al. (2021) in its entirety but with the different procedures calibrated to have the same predetermined maximum Type I error rate. Our results suggest that, when calibrated in this way, the Bayes factor, HDI-ROPE, and frequentist equivalence tests all have similar-almost exactly-Type II error rates. In general any advocating for frequentist testing as better or worse than Bayesian testing in terms of empirical findings seems dubious at best. If one decides on which underlying principle to subscribe to in tackling a given problem, then the method follows naturally. Bearing in mind that each procedure can be reverse-engineered from the others (at least approximately), trying to use empirical performance to argue for 1 approach over another seems like tilting at windmills. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

How the Post-Data Severity Converts Testing Results into Evidence for or against Pertinent Inferential Claims.

The paper makes a case that the current discussions on replicability and the abuse of significance testing have overlooked a more general contributor to the untrustworthiness of published empirical evidence, which is the uninformed and recipe-like implementation of statistical modeling and inference. It is argued that this contributes to the untrustworthiness problem in several different ways, including [a] statistical misspecification, [b] unwarranted evidential interpretations of frequentist inference results, and [c] questionable modeling strategies that rely on curve-fitting. What is more, the alternative proposals to replace or modify frequentist testing, including [i] replacing p-values with observed confidence intervals and effects sizes, and [ii] redefining statistical significance, will not address the untrustworthiness of evidence problem since they are equally vulnerable to [a]-[c]. The paper calls for distinguishing between unduly data-dependant 'statistical results', such as a point estimate, a p-value, and accept/reject H0, from 'evidence for or against inferential claims'. The post-data severity (SEV) evaluation of the accept/reject H0 results, converts them into evidence for or against germane inferential claims. These claims can be used to address/elucidate several foundational issues, including (i) statistical vs. substantive significance, (ii) the large n problem, and (iii) the replicability of evidence. Also, the SEV perspective sheds light on the impertinence of the proposed alternatives [i]-[iii], and oppugns [iii] the alleged arbitrariness of framing H0 and H1 which is often exploited to undermine the credibility of frequentist testing.

Revisiting the Large n (Sample Size) Problem: How to Avert Spurious Significance Results

Although large data sets are generally viewed as advantageous for their ability to provide more precise and reliable evidence, it is often overlooked that these benefits are contingent upon certain conditions being met. The primary condition is the approximate validity (statistical adequacy) of the probabilistic assumptions comprising the statistical model Mθ(x) applied to the data. In the case of a statistically adequate Mθ(x) and a given significance level α, as n increases, the power of a test increases, and the p-value decreases due to the inherent trade-off between type I and type II error probabilities in frequentist testing. This trade-off raises concerns about the reliability of declaring ‘statistical significance’ based on conventional significance levels when n is exceptionally large. To address this issue, the author proposes that a principled approach, in the form of post-data severity (SEV) evaluation, be employed. The SEV evaluation represents a post-data error probability that converts unduly data-specific ‘accept/reject H0 results’ into evidence either supporting or contradicting inferential claims regarding the parameters of interest. This approach offers a more nuanced and robust perspective in navigating the challenges posed by the large n problem.

The NANOGrav 15 yr Data Set: Evidence for a Gravitational-wave Background

We report multiple lines of evidence for a stochastic signal that is correlated among 67 pulsars from the 15 yr pulsar timing data set collected by the North American Nanohertz Observatory for Gravitational Waves. The correlations follow the Hellings–Downs pattern expected for a stochastic gravitational-wave background. The presence of such a gravitational-wave background with a power-law spectrum is favored over a model with only independent pulsar noises with a Bayes factor in excess of 1014, and this same model is favored over an uncorrelated common power-law spectrum model with Bayes factors of 200–1000, depending on spectral modeling choices. We have built a statistical background distribution for the latter Bayes factors using a method that removes interpulsar correlations from our data set, finding p = 10−3 (≈3σ) for the observed Bayes factors in the null no-correlation scenario. A frequentist test statistic built directly as a weighted sum of interpulsar correlations yields p = 5 × 10−5 to 1.9 × 10−4 (≈3.5σ–4σ). Assuming a fiducial f −2/3 characteristic strain spectrum, as appropriate for an ensemble of binary supermassive black hole inspirals, the strain amplitude is (median + 90% credible interval) at a reference frequency of 1 yr−1. The inferred gravitational-wave background amplitude and spectrum are consistent with astrophysical expectations for a signal from a population of supermassive black hole binaries, although more exotic cosmological and astrophysical sources cannot be excluded. The observation of Hellings–Downs correlations points to the gravitational-wave origin of this signal.

Evaluation of frequentist test statistics using constrained statistical inference in the context of the generalized linear model

ABSTRACT When faced with a binary or count outcome, informative hypotheses can be tested in the generalized linear model using the distance statistic as well as modified versions of the Wald, the Score and the likelihood-ratio test (LRT). In contrast to classical null hypothesis testing, informative hypotheses allow to directly examine the direction or the order of the regression coefficients. Since knowledge about the practical performance of informative test statistics is missing in the theoretically oriented literature, we aim at closing this gap using simulation studies in the context of logistic and Poisson regression. We examine the effect of the number of constraints as well as the sample size on type I error rates when the hypothesis of interest can be expressed as a linear function of the regression parameters. The LRT shows the best performance in general, followed by the Score test. Furthermore, both the sample size and especially the number of constraints impact the type I error rates considerably more in logistic compared to Poisson regression. We provide an empirical data example together with R code that can be easily adapted by applied researchers. Moreover, we discuss informative hypothesis testing about effects of interest, which are a non-linear function of the regression parameters. We demonstrate this by means of a second empirical data example.

Ponderings on the Possible Preponderance of Perpendicular Planets

Misalignments between planetary orbits and the equatorial planes of their host stars are clues about the formation and evolution of planetary systems. Earlier work found evidence for a peak near 90° in the distribution of stellar obliquities, based on frequentist tests. We performed hierarchical Bayesian inference on a sample of 174 planets for which either the full three-dimensional stellar obliquity has been measured (72 planets) or for which only the sky-projected stellar obliquity has been measured (102 planets). We investigated whether the obliquities are best described by a Rayleigh distribution or by a mixture of a Rayleigh distribution representing well-aligned systems and a different distribution representing misaligned systems. The mixture models are strongly favored over the single-component distribution. For the misaligned component, we tried an isotropic distribution and a distribution peaked at 90° and found the evidence to be essentially the same for both models. Thus, our Bayesian inference engine did not find strong evidence favoring a “perpendicular peak,” unlike the frequentist tests. We also investigated selection biases that affect the inferred obliquity distribution, such as the bias of the gravity-darkening method against obliquities near 0° or 180°. Further progress in characterizing the obliquity distribution will probably require the construction of a more homogeneous and complete sample of measurements.

The e-posterior.

We develop a representation of a decision maker's uncertainty based on e-variables. Like the Bayesian posterior, this e-posterior allows for making predictions against arbitrary loss functions that may not be specified ex ante. Unlike the Bayesian posterior, it provides risk bounds that have frequentist validity irrespective of prior adequacy: if the e-collection (which plays a role analogous to the Bayesian prior) is chosen badly, the bounds get loose rather than wrong, making e-posterior minimax decision rules safer than Bayesian ones. The resulting quasi-conditional paradigm is illustrated by re-interpreting a previous influential partial Bayes-frequentist unification, Kiefer-Berger-Brown-Wolpert conditional frequentist tests, in terms of e-posteriors. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

Identifying the Influencing Factors for the BMI by Bayesian and Frequentist Multiple Linear Regression Models: A Comparative Study.

In this article, we attempt to demonstrate the superiority of the Bayesian approach over the frequentist approaches of the multiple linear regression model in identifying the influencing factors for the response variable. A survey was conducted among the 310 respondents from the Kathirkamam area in Puducherry. We have considered the response variable, body mass index (BMI), and the predictors such as age, weight, gender, nature of the job, and marital status of individuals were collected with the personal interview method. Jeffreys's Amazing Statistics Program (JASP) software was used to analyze the dataset. In the conventional multiple linear regression model, the single value of regression coefficients is determined, while in the Bayesian linear regression model, the regression coefficient of each predictor follows a specific posterior distribution. Furthermore, it would be most useful to identify the best models from the list of possible models with posterior probability values. An inclusion probability for all the predictors will give a superior idea of whether the predictors are included in the model with probability. The Bayesian framework offers a wide range of results for the regression coefficients instead of the single value of regression coefficients in the frequentist test. Such advantages of the Bayesian approach will catapult the quality of investigation outputs by giving more reliability to solutions of scientific problems.

Why Bayesian analysis of clinical trials may be a good idea– a worked example of the aducanumab studies

AbstractBackgroundThe FDA decision to approve Aducanumab has sparked controversy. Here, we argue that some of these controversies are rooted in the conclusions facilitated by classical frequentist analysis. We suggest that a Bayesian analysis would answer some of the burning questions the aducanumab trials have posed to the field.MethodWe applied Bayesian analysis of model plausibility and effect sizes based on simulated data (the original data are not publicly available) of the two phase 3 trials (trials 301 and 302) of aducanumab in prodromal and mild dementia stages of Alzheimer’s disease.ResultExcluding the rapid progressors from our simulated data did not produce decisive evidence supporting aducanumab’s treatment effect. The Bayesian analysis suggested that the study indicator (study 301 or study 302) was irrelevant as a predictor of treatment effect (Bayes Factor (BF)exc = 42.16). Our simulated data further offered overwhelming evidence against Biogen’s assumption that rapid progressors affected any treatment arm of either study disproportionately (i.e., BFexc = 9910.74). We also found anecdotal evidence against the treatment arms (i.e., BFexc = 2.65) as a predictor of CDR‐SB change (Figure 1). The prior and posterior plots show that if the effect existed, it would be very small, likely ranging from Kendall’s tau 0.007 to 0.052, with a median of 0.029 (Figure 2).ConclusionBayesian analysis can quantify evidence in favor of the presence or absence of an effect, as well as assess the strength of the effect. In the case of our simulated data, Bayesian analysis quantified evidence that studies 301 and 302 do not provide diverging evidence, and that the treatment arm‐specific rapid progressor effect was absent. The Bayesian analysis thus provides more informative and nuanced conclusions than the reported frequentist tests, which leave many questions unanswered.

Interpreting p-Values and Confidence Intervals Using Well-Calibrated Null Preference Priors

We propose well-calibrated null preference priors for use with one-sided hypothesis tests, such that resulting Bayesian and frequentist inferences agree. Null preference priors mean that they have nearly 100% of their prior belief in the null hypothesis, and well-calibrated priors mean that the resulting posterior beliefs in the alternative hypothesis are not overconfident. This formulation expands the class of problems giving Bayes-frequentist agreement to include problems involving discrete distributions such as binomial and negative binomial one- and two-sample exact (i.e., valid) tests. When applicable, these priors give posterior belief in the null hypothesis that is a valid p-value, and the null preference prior emphasizes that large p-values may simply represent insufficient data to overturn prior belief. This formulation gives a Bayesian interpretation of some common frequentist tests, as well as more intuitively explaining lesser known and less straightforward confidence intervals for two-sample tests.

Overview and evaluation of various frequentist test statistics using constrained statistical inference in the context of linear regression.

Within the framework of constrained statistical inference, we can test informative hypotheses, in which, for example, regression coefficients are constrained to have a certain direction or be in a specific order. A large amount of frequentist informative test statistics exist that each come with different versions, strengths and weaknesses. This paper gives an overview about these statistics, including the Wald, the LRT, the Score, the - and the D-statistic. Simulation studies are presented that clarify their performance in terms of type I and type II error rates under different conditions. Based on the results, it is recommended to use the Wald and -test rather than the LRT and Score test as the former need less computing time. Furthermore, it is favorable to use the degrees of freedom corrected rather than the naive mean squared error when calculating the test statistics as well as using the - rather than the -distribution when calculating the p-values.

Strong-Form Frequentist Testing In Communication Science: Principles, Opportunities, And Challenges

ABSTRACT This paper discusses ‘strong-form’ frequentist testing as a useful complement to null hypothesis testing in communication science. In a ‘strong-form’ set-up a researcher defines a hypothetical effect size of (minimal) theoretical interest and assesses to what extent her findings falsify or corroborate that particular hypothesis. We argue that the idea of ‘strong-form’ testing aligns closely with the ideals of the movements for scientific reform, discuss its technical application within the context of the General Linear Model, and show how the relevant P-value-like quantities can be calculated and interpreted. We also provide examples and a simulation to illustrate how a strong-form set-up requires more nuanced reflections about research findings. In addition, we discuss some pitfalls that might still hold back strong-form tests from widespread adoption.

Bayesian Approach: Adding Clinical Edge in Interpreting Medical Data

In frequentist tests, the significance testing framework for null hypothesis permits dichotomous conclusions alone, and such tests do not quantify the strength of the evidence supporting the null hypothesis. Under the Bayesian approach, probability reflects their uncertainty or degree of belief, that is, how scientific belief should be modified by data. This paper attempts to demonstrate the advantages of the Bayes factor in hypothesis testing that can quantify evidence in favour of the null hypothesis and how the prior specification is used for statistical tools, such as independent t-test and Analysis of Variance (ANOVA). Despite the advantages of the Bayesian approach, the use of conventional tests that rely on inference by p-values is ubiquitous in medical research. The adoption of the Bayesian approach may be seriously hindered by the absence of formulae, algorithms, etc. Furthermore, we have attempted to validate our argument by interpreting the application of both the Frequentist and Bayesian approaches for dietary intake of calcium mg/day with the help of JASP software.

A Bayesian perspective on Biogen's aducanumab trial.

This perspective is a companion to a recent editorial on the use of Bayesian analysis in clinical research. We aim to introduce and highlight the relevance and advantages that Bayesian inference offers to clinical trials using the data on the amyloid antibody aducanumab presented at a Food and Drug Administration hearing in November 2020 as an applied example. We apply Bayesian analysis of model plausibility and effect sizes based on simulated data of the two phase 3 trials of aducanumab in prodromal and mild dementia stages of Alzheimer's disease (AD). Bayesian analysis can quantify evidence in favor of, or against, the presence of an effect (i.e., provide evidence of absence), as well as assess the strength of the effect. This is in contrast to the binary conclusions provided by frequentist tests.

Severity and Trustworthy Evidence: Foundational Problems versus Misuses of Frequentist Testing

AbstractFor model-based frequentist statistics, based on a parametric statistical model${{\cal M}_\theta }({\bf{x}})$, the trustworthiness of the ensuing evidence depends crucially on (i) the validity of the probabilistic assumptions comprising${{\cal M}_\theta }({\bf{x}})$, (ii) the optimality of the inference procedures employed, and (iii) the adequateness of the sample size (n) to learn from data by securing (i)–(ii). It is argued that the criticism of the postdata severity evaluation of testing results based on a smallnby Rochefort-Maranda (2020) is meritless because it conflates [a] misuses of testing with [b] genuine foundational problems. Interrogating this criticism reveals several misconceptions about trustworthy evidence and estimation-based effect sizes, which are uncritically embraced by the replication crisis literature.

Frequentist test in Bayesian two-stage designs applied in experimental trials

Prediction provides discipline and pragmatic importance to empirical research. The design with the predictive probability approach provides an excellent alternative for conducting multi-stage phase II trials; it is efficient and flexible and possesses desirable statistical properties. . In this paper we consider the Bayesian predictive procedures within the experimental design, for this, we define indices of satisfaction related to a test as a decreasing function of the p-value and satisfaction is higher than the null hypothesis is rejected wider. This design possesses good frequentist properties and allows early termination of the trial. We treated our applications by simulation and real data in experimental planning and sequential designs with binary outcomes.

Optimal Decision Rules for Weak GMM

This paper studies optimal decision rules, including estimators and tests, for weakly identified GMM models. We derive the limit experiment for weakly identified GMM, and propose a theoretically‐motivated class of priors which give rise to quasi‐Bayes decision rules as a limiting case. Together with results in the previous literature, this establishes desirable properties for the quasi‐Bayes approach regardless of model identification status, and we recommend quasi‐Bayes for settings where identification is a concern. We further propose weighted average power‐optimal identification‐robust frequentist tests and confidence sets, and prove a Bernstein‐von Mises‐type result for the quasi‐Bayes posterior under weak identification.

A Hybrid Monitoring Procedure for Detecting Abnormality with Application to Energy Consumption Data

The complexity of energy infrastructure at large institutions increasingly calls for data-driven monitoring of energy usage. This article presents a hybrid monitoring algorithm for detecting consumption surges using statistical hypothesis testing, leveraging the posterior distribution and its information about uncertainty to introduce randomness in the parameter estimates, while retaining the frequentist testing framework. This hybrid approach is designed to be asymptotically equivalent to the Neyman-Pearson test. We show via extensive simulation studies that the hybrid approach enjoys control over type-1 error rate even with finite sample sizes whereas the naive plug-in method tends to exceed the specified level, resulting in overpowered tests. The proposed method is applied to the natural gas usage data at the University of Connecticut.