Hypothesis Testing Procedures Research Articles

Design of randomized clinical trials with a binary endpoint: Conditional versus unconditional analyses of a two-by-two table.

When designing a randomized clinical trial to compare two treatments, the sample size required to have desired power with a specified type 1 error depends on the hypothesis testing procedure. With a binary endpoint (e.g., response), the trial results can be displayed in a 2 × 2 table. If one does the analysis conditional on the number of positive responses, then using Fisher's exact test has an actual type 1 error less than or equal to the specified nominal type 1 error. Alternatively, one can use one of many unconditional "exact" tests that also preserve the type 1 error and are less conservative than Fisher's exact test. In particular, the unconditional test of Boschloo is always at least as powerful as Fisher's exact test, leading to smaller required sample sizes for clinical trials. However, many statisticians have argued over the years that the conditional analysis with Fisher's exact test is the only appropriate procedure. Since having smaller clinical trials is an extremely important consideration, we review the general arguments given for the conditional analysis of a 2 × 2 table in the context of a randomized clinical trial. We find the arguments not relevant in this context, or, if relevant, not completely convincing, suggesting the sample-size advantage of the unconditional tests should lead to their recommended use. We also briefly suggest that since designers of clinical trials practically always have target null and alternative response rates, there is the possibility of using this information to improve the power of the unconditional tests.

Detecting Weak Distribution Shifts via Displacement Interpolation

Detecting weak, systematic distribution shifts and quantitatively modeling individual, heterogeneous responses to policies or incentives have found increasing empirical applications in social and economic sciences. Given two probability distributions P (null) and Q (alternative), we study the problem of detecting weak distribution shift deviating from the null P toward the alternative Q, where the level of deviation vanishes as a function of n, the sample size. We propose a model for weak distribution shifts via displacement interpolation between P and Q, drawing from the optimal transport theory. We study a hypothesis testing procedure based on the Wasserstein distance, derive sharp conditions under which detection is possible, and provide the exact characterization of the asymptotic Type I and Type II errors at the detection boundary using empirical processes. We demonstrate how the proposed testing procedure works in modeling and detecting weak distribution shifts in real datasets using two empirical examples: distribution shifts in consumer spending after COVID-19, and heterogeneity in the published p-values of statistical tests in journals across different disciplines.

A Plot is Worth a Thousand Tests: Assessing Residual Diagnostics with the Lineup Protocol

Regression experts consistently recommend plotting residuals for model diagnosis, despite the availability of many numerical hypothesis test procedures designed to use residuals to assess problems with a model fit. Here we provide evidence for why this is good advice using data from a visual inference experiment. We show how conventional tests are too sensitive, which means that too often the conclusion would be that the model fit is inadequate. The experiment uses the lineup protocol which puts a residual plot in the context of null plots. This helps generate reliable and consistent reading of residual plots for better model diagnosis. It can also help in an obverse situation where a conventional test would fail to detect a problem with a model due to contaminated data. The lineup protocol also detects a range of departures from good residuals simultaneously. Supplemental materials for the article are available online.

Efficient alternatives for Bayesian hypothesis tests in psychology.

Bayesian hypothesis testing procedures have gained increased acceptance in recent years. A key advantage that Bayesian tests have over classical testing procedures is their potential to quantify information in support of true null hypotheses. Ironically, default implementations of Bayesian tests prevent the accumulation of strong evidence in favor of true null hypotheses because associated default alternative hypotheses assign a high probability to data that are most consistent with a null effect. We propose the use of "nonlocal" alternative hypotheses to resolve this paradox. The resulting class of Bayesian hypothesis tests permits more rapid accumulation of evidence in favor of both true null hypotheses and alternative hypotheses that are compatible with standardized effect sizes of most interest in psychology. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihood

In supervised learning, automatically assessing the quality of the labels before any learning takes place remains an open research question. In certain particular cases, hypothesis testing procedures have been proposed to assess whether a given instance-label dataset is contaminated with class-conditional label noise, as opposed to uniform label noise. The existing theory builds on the asymptotic properties of the Maximum Likelihood Estimate for parametric logistic regression. However, the parametric assumptions on top of which these approaches are constructed are often too strong and unrealistic in practice. To alleviate this problem, in this paper we propose an alternative path by showing how similar procedures can be followed when the underlying model is a product of Local Maximum Likelihood Estimation that leads to more flexible nonparametric logistic regression models, which in turn are less susceptible to model misspecification. This different view allows for wider applicability of the tests by offering users access to a richer model class. Similarly to existing works, we assume we have access to anchor points which are provided by the users. We introduce the necessary ingredients for the adaptation of the hypothesis tests to the case of nonparametric logistic regression and empirically compare against the parametric approach presenting both synthetic and real-world case studies and discussing the advantages and limitations of the proposed approach.

Sequential hypothesis testing for selecting the number of changepoints in segmented regression models

AbstractSegmented regression is widely used in many disciplines, especially when dealing with environmental data. This paper deals with the problem of selecting the correct number of changepoints in segmented regression models. A review of the usual selection criteria, namely information criteria and hypothesis testing, is provided. We enhance the latter method by proposing a novel sequential hypothesis testing procedure to address this problem. Our sequential procedure’s performance is compared to methods based on information-based criteria through simulation studies. The results show that our proposal performs similarly to its competitors for the Gaussian, Binomial, and Poisson cases. Finally, we present two applications to environmental datasets of crime data in Valencia and global temperature land data.

A new Bayesian discrepancy measure

AbstractThe aim of this article is to make a contribution to the Bayesian procedure of testing precise hypotheses for parametric models. For this purpose, we define the Bayesian Discrepancy Measure that allows one to evaluate the suitability of a given hypothesis with respect to the available information (prior law and data). To summarise this information, the posterior median is employed, allowing a simple assessment of the discrepancy with a fixed hypothesis. The Bayesian Discrepancy Measure assesses the compatibility of a single hypothesis with the observed data, as opposed to the more common comparative approach where a hypothesis is rejected in favour of a competing hypothesis. The proposed measure of evidence has properties of consistency and invariance. After presenting the definition of the measure for a parameter of interest, both in the absence and in the presence of nuisance parameters, we illustrate some examples showing its conceptual and interpretative simplicity. Finally, we compare a test procedure based on the Bayesian Discrepancy Measure, with the Full Bayesian Significance Test, a well-known Bayesian testing procedure for sharp hypotheses.

Suboptimal Diets Identified Among Adults in Two Rural States During the COVID-19 Pandemic

ABSTRACT Rural communities face unique food and nutrition challenges. Analyzing population-level survey data from 2021 to 2022 in two predominantly rural U.S. states using regression and multiple hypothesis testing procedures, we examined how well diets align with national recommendations and associations between diet, age, food security, and rurality. We found dietary intake of key foods and ingredients to differ significantly from recommendations and overall diet quality to increase with age. Food secure adults had higher diet quality than food insecure adults. Neither dietary intake nor dietary quality differed by degree of rurality. Continued action is needed to address nutrition challenges among underserved populations.

Extending Buckley–James method for heteroscedastic survival data

The Buckley–James method for the classical accelerated failure time model has been extended to accommodate heteroscedastic survival data in two ways. The first is the weighted least squares method [Yu et al. Weighted least-squares method for right-censored data in accelerated failure time model. Biometrics. 2013;69:358–365], which estimates the heteroscedasticity nonparametrically, while the second is the local Buckley–James method [Pang et al. Local Buckley–James estimation for heteroscedastic accelerated failure time model. Stat Sin. 2015;25:863–877], which uses local Kaplan–Meier method to estimate the heteroscedasticity. However, no comparisons have been done for these two methods. Furthermore, there is no hypothesis testing procedure for this heteroscedastic accelerated failure time model. This paper is then aimed to fill these two gaps to compare the two methods theoretically and numerically with extensive simulation studies. In addition, we propose a class of hypothesis tests for the parameters to provide a complete procedure for analysing heteroscedastic survival data. Two real data examples are used for practical illustration of the comparison and the new proposed tests.

Factors Influencing Pet Attachment: Exploring Attachment Dimensions and Demographic Factors

This study examined the factors influencing pet attachment by investigating attachment dimensions and exploring the relationship between demographic factors and pet attachment. The study utilized the Pet Attachment Questionnaire (PAQ) to assess the level of attachment between pet owners and their animals. A demographic questionnaire was also administered to gather socio-cultural, economic, and health-related data from pet caretakers. A confirmatory factor analysis was applied to confirm the scale factor structure. Hypothesis testing procedures were used to reveal the relationship between the demographic characteristics of the participants and the attachment relationships. The study involved 304 volunteers who visited the animal hospital at Ankara University, Faculty of Veterinary Medicine. The findings revealed significant impacts of various factors on attachment dimensions, including age, household income, participant and household member anxieties and traumas, number of pets owned, pet health, and previous pet ownership. These results contributed to our understanding of the complex dynamics that shape the attachment between humans and animals. Further research is needed to delve into the underlying mechanisms and potential interactions among these factors, advancing our knowledge of human-pet attachment.

Competing Risk Modeling with Bivariate Varying Coefficients to Understand the Dynamic Impact of COVID-19

The coronavirus disease 2019 (COVID-19) pandemic has exerted a profound impact on patients with end-stage renal disease relying on kidney dialysis to sustain their lives. A preliminary analysis of dialysis patient postdischarge hospital readmissions and deaths in 2020 revealed that the COVID-19 effect has varied significantly with postdischarge time and time since the pandemic onset. However, the complex dynamics cannot be characterized by existing varying coefficient models. To address this issue, we propose a bivariate varying coefficient model for competing risks, where tensor-product B-splines are used to estimate the surface of the COVID-19 effect. An efficient proximal Newton algorithm is developed to facilitate the fitting of the new model to the massive data for Medicare beneficiaries on dialysis. Difference-based anisotropic penalization is introduced to mitigate model overfitting and effect wiggliness; a cross-validation method is derived to determine optimal tuning parameters. Hypothesis testing procedures are designed to examine whether the COVID-19 effect varies significantly with postdischarge time and the time since the pandemic onset, either jointly or separately. Applications to Medicare dialysis patients demonstrate the real-world performance of the proposed methods. Simulation experiments are conducted to evaluate the estimation accuracy, type I error rate, statistical power, and model selection procedures. Supplementary materials for this article are available online.

Variance of entropy for testing time-varying regimes with an application to meme stocks

AbstractShannon entropy is the most common metric for assessing the degree of randomness of time series in many fields, ranging from physics and finance to medicine and biology. Real-world systems are typically non-stationary, leading to entropy values fluctuating over time. This paper proposes a hypothesis testing procedure to test the null hypothesis of constant Shannon entropy in time series data. The alternative hypothesis is a significant variation in entropy between successive periods. To this end, we derive an unbiased sample entropy variance, accurate up to the order $$O(n^{-4})$$ O ( n - 4 ) with n the sample size. To characterize the variance of the sample entropy, we first provide explicit formulas for the central moments of both binomial and multinomial distributions describing the distribution of the sample entropy. Second, we identify the optimal rolling window length to estimate time-varying Shannon entropy. We optimize this choice using a novel self-consistent criterion based on counting significant entropy variations over time. We corroborate our findings using the novel methodology to assess time-varying regimes of entropy for stock price dynamics by presenting a comparative analysis between meme and IT stocks in 2020 and 2021. We show that low entropy values correspond to periods when profitable trading strategies can be devised starting from the symbolic dynamics used for entropy computation, namely periods of market inefficiency.

PERCEIVED ORGANISATIONAL SUPPORT AND ORGANISATIONAL TRUST LINK TO INNOVATIVE WORK BEHAVIOUR BY THE MEDIATION OF TACIT KNOWLEDGE SHARING

The aim of this study is to determine how perceived organisational support and organisational trust affect innovative work behaviour through the mediation mechanism of tacit knowledge sharing in a public organisation context in Nigeria. Using the cross-sectional research design, data were randomly collected from 258 employees from the Delta State Ministry of Health, which was subjected to the partial least square (PLS) analytical procedure for hypothesis testing. The PLS analysis results suggested that while tacit knowledge sharing mediates perceived organisational support and organisational trust link to innovative work behaviour, it also mediates perceived organisational support link to innovative work behaviour sequentially with organisational trust as an antecedent. The study concluded that perceived organisational support and organisational trust prediction of innovative work behaviour can be explained by tacit knowledge sharing. The study puts forward important practical implications for organisations/management to follow in effectuating the linkages among the constructs in a meaningful and coherent manner.

Genome-wide search algorithms for identifying dynamic gene co-expression via Bayesian variable selection.

A wealth of gene expression data generated by high-throughput techniques provides exciting opportunities for studying gene-gene interactions systematically. Gene-gene interactions in a biological system are tightly regulated and are often highly dynamic. The interactions can change flexibly under various internal cellular signals or external stimuli. Previous studies have developed statistical methods to examine these dynamic changes in gene-gene interactions. However, due to the massive number of possible gene combinations that need to be considered in a typical genomic dataset, intensive computation is a common challenge for exploring gene-gene interactions. On the other hand, oftentimes only a small proportion of gene combinations exhibit dynamic co-expression changes. To solve this problem, we propose Bayesian variable selection approaches based on spike-and-slab priors. The proposed algorithms reduce the computational intensity by focusing on identifying subsets of promising gene combinations in the search space. We also adopt a Bayesian multiple hypothesis testing procedure to identify strong dynamic gene co-expression changes. Simulation studies are performed to compare the proposed approaches with existing exhaustive search heuristics. We demonstrate the implementation of our proposed approach to study the association between gene co-expression patterns and overall survival using the RNA-sequencing dataset from The Cancer Genome Atlas breast cancer BRCA-US project.

Granger causality tests based on reduced variable information

Granger causality is a classical and important technique for measuring predictability from one group of time series to another by incorporating information of the variables described by a full vector autoregressive (VAR) process. However, in some applications economic forecasts need to be made based on information provided merely by a portion of variates (e.g., removal of a listed stock due to halting, suspension or delisting). This requires a new formulation of forecast based on an embedded subprocess of VAR, whose theoretical properties are often difficult to obtain. To avoid the issue of identifying the VAR subprocess, we propose a computation‐based approach so that sophisticated predictions can be made by utilizing a reduced variable information set estimated from sampled data. Such estimated information set allows us to develop a suitable statistical hypothesis testing procedure for characterizing all designated Granger causal relationships, as well as a useful graphical tool for presenting the causal structure over the prediction horizon. Finally, simulated data and a real example from the stock markets are used to illustrate the proposed method.

Hypothesis testing procedure for binary and multi-class F1 -scores in the paired design.

In modern medicine, medical tests are used for various purposes including diagnosis, disease screening, prognosis, and risk prediction. To quantify the performance of the binary medical test, we often use sensitivity, specificity, and negative and positive predictive values as measures. Additionally, the -score, which is defined as the harmonic mean of precision (positive predictive value) and recall (sensitivity), has come to be used in the medical field due to its favorable characteristics. The -score has been extended for multi-class classification, and two types of -scores have been proposed for multi-class classification: a micro-averaged -score and a macro-averaged -score. The micro-averaged -score pools per-sample classifications across classes and then calculates the overall -score, whereas the macro-averaged -score computes an arithmetic mean of the -scores for each class. Additionally, Sokolova and Lapalme gave an alternative definition of the macro-averaged -score as the harmonic mean of the arithmetic means of the precision and recall over classes. Although some statistical methods of inference for binary and multi-class -scores have been proposed, the methodology development of hypothesis testing procedure for them has not been fully progressing yet. Therefore, we aim to develop hypothesis testing procedure for comparing two -scores in paired study design based on the large sample multivariate central limit theorem.

A Bayesian actor-oriented multilevel relational event model with hypothesis testing procedures

AbstractRelational event network data are becoming increasingly available. Consequently, statistical models for such data have also surfaced. These models mainly focus on the analysis of single networks; while in many applications, multiple independent event sequences are observed, which are likely to display similar social interaction dynamics. Furthermore, statistical methods for testing hypotheses about social interaction behavior are underdeveloped. Therefore, the contribution of the current paper is twofold. First, we present a multilevel extension of the dynamic actor-oriented model, which allows researchers to model sender and receiver processes separately. The multilevel formulation enables principled probabilistic borrowing of information across networks to accurately estimate drivers of social dynamics. Second, a flexible methodology is proposed to test hypotheses about common and heterogeneous social interaction drivers across relational event sequences. Social interaction data between children and teachers in classrooms are used to showcase the methodology.

Effect of the U.S.–China Trade War on Stock Markets: A Financial Contagion Perspective

In this article, to model risk contagion between the U.S. and China stock markets based on high-frequency financial data, we develop a novel continuous-time jump-diffusion process. For example, we consider three channels for volatility contagion—such as integrated volatility, positive jump variation, and negative jump variation—and each stock market is able to affect the other stock market as an overnight risk factor. We develop a quasi-maximum likelihood estimator for model parameters and establish its asymptotic properties. Furthermore, to identify contagion channels and test the existence of a structural break with a known structural break date, we propose hypothesis test procedures. Using the proposed diffusion model with high-frequency financial data, we investigate the effect of the U.S.–China trade war on stock markets from a financial contagion perspective. From the empirical study, we find evidence of financial contagion from the United States to China and evidence that the risk contagion channel has changed from integrated volatility to negative jump variation.

Characterization of direct and/or indirect genetic associations for multiple traits in longitudinal studies of disease progression.

When quantitative longitudinal traits are risk factors for disease progression and subject to random biological variation, joint model analysis of time-to-event and longitudinal traits can effectively identify direct and/or indirect genetic association of single nucleotide polymorphisms (SNPs) with time-to-event. We present a joint model that integrates: (1) a multivariate linear mixed model describing trajectories of multiple longitudinal traits as a function of time, SNP effects, and subject-specific random effects and (2) a frailty Cox survival model that depends on SNPs, longitudinal trajectory effects, and subject-specific frailty accounting for dependence among multiple time-to-event traits. Motivated by complex genetic architecture of type 1 diabetes complications (T1DC) observed in the Diabetes Control and Complications Trial (DCCT), we implement a 2-stage approach to inference with bootstrap joint covariance estimation and develop a hypothesis testing procedure to classify direct and/or indirect SNP association with each time-to-event trait. By realistic simulation study, we show that joint modeling of 2 time-to-T1DC (retinopathy and nephropathy) and 2 longitudinal risk factors (HbA1c and systolic blood pressure) reduces estimation bias in genetic effects and improves classification accuracy of direct and/or indirect SNP associations, compared to methods that ignore within-subject risk factor variability and dependence among longitudinal and time-to-event traits. Through DCCT data analysis, we demonstrate feasibility for candidate SNP modeling and quantify effects of sample size and Winner's curse bias on classification for 2 SNPs identified as having indirect associations with time-to-T1DC traits. Joint analysis of multiple longitudinal and multiple time-to-event traits provides insight into complex traits architecture.

A mark-specific quantile regression model

Summary Quantile regression has become a widely used tool for analysing competing risk data. However, quantile regression for competing risk data with a continuous mark is still scarce. The mark variable is an extension of cause of failure in a classical competing risk model where cause of failure is replaced by a continuous mark only observed at uncensored failure times. An example of the continuous mark variable is the genetic distance that measures dissimilarity between the infecting virus and the virus contained in the vaccine construct. In this article, we propose a novel mark-specific quantile regression model. The proposed estimation method borrows strength from data in a neighbourhood of a mark and is based on an induced smoothed estimation equation, which is very different from the existing methods for competing risk data with discrete causes. The asymptotic properties of the resulting estimators are established across mark and quantile continuums. In addition, a mark-specific quantile-type vaccine efficacy is proposed and its statistical inference procedures are developed. Simulation studies are conducted to evaluate the finite sample performances of the proposed estimation and hypothesis testing procedures. An application to the first HIV vaccine efficacy trial is provided.