Conventional Hypothesis Testing Research Articles

Testing hypotheses under adaptive randomization with continuous covariates in clinical trials.

Covariate-adaptive designs are widely used to balance covariates and maintain randomization in clinical trials. Adaptive designs for discrete covariates and their asymptotic properties have been well studied in the literature. However, important continuous covariates are often involved in clinical studies. Simply discretizing or categorizing continuous covariates can result in loss of information. The current understanding of adaptive designs with continuous covariates lacks a theoretical foundation as the existing works are entirely based on simulations. Consequently, conventional hypothesis testing in clinical trials using continuous covariates is still not well understood. In this paper, we establish a theoretical framework for hypothesis testing on adaptive designs with continuous covariates based on linear models. For testing treatment effects and significance of covariates, we obtain the asymptotic distributions of the test statistic under null and alternative hypotheses. Simulation studies are conducted under a class of covariate-adaptive designs, including the p-value-based method, the Su's percentile method, the empirical cumulative-distribution method, the Kullback-Leibler divergence method, and the kernel-density method. Key findings about adaptive designs with independent covariates based on linear models are (1) hypothesis testing that compares treatment effects are conservative in terms of smaller type I error, (2) hypothesis testing using adaptive designs outperforms complete randomization method in terms of power, and (3) testing on significance of covariates is still valid.

Energy consumption and gross domestic product in the Philippines: an application of maximum entropy bootstrap framework

We employ the maximum entropy bootstrap (MEB) framework to provide convincing evidence on the energy consumption (EC) and gross domestic product (GDP) nexus between 1975 and 2010 in the Philippines. We also perform a cointegration analysis and Granger causality test of the data to illustrate the advantages of MEB approach. This paper shows more accurate inference in comparison to conventional hypothesis tests based on asymptotic theory. Without employing MEB, the result of causality is very sensitive to a very small sample size and time period chosen that made the results inconsistent. Therefore, MEB framework is robust to time period chosen and even in a very small sample size. The analysis shows no evidence of a causal relationship between EC and GDP in the Philippines. The findings emphasise the fact that the Philippines is a less developed country and predominantly agrarian-based; thus, energy dependent.

Energy consumption and gross domestic product in the Philippines: an application of maximum entropy bootstrap framework

We employ the maximum entropy bootstrap (MEB) framework to provide convincing evidence on the energy consumption (EC) and gross domestic product (GDP) nexus between 1975 and 2010 in the Philippines. We also perform a cointegration analysis and Granger causality test of the data to illustrate the advantages of MEB approach. This paper shows more accurate inference in comparison to conventional hypothesis tests based on asymptotic theory. Without employing MEB, the result of causality is very sensitive to a very small sample size and time period chosen that made the results inconsistent. Therefore, MEB framework is robust to time period chosen and even in a very small sample size. The analysis shows no evidence of a causal relationship between EC and GDP in the Philippines. The findings emphasise the fact that the Philippines is a less developed country and predominantly agrarian-based; thus, energy dependent.

Modeling and replicating statistical topology and evidence for CMB nonhomogeneity

Under the banner of "big data," the detection and classification of structure in extremely large, high-dimensional, data sets are two of the central statistical challenges of our times. Among the most intriguing new approaches to this challenge is "TDA," or "topological data analysis," one of the primary aims of which is providing nonmetric, but topologically informative, preanalyses of data which make later, more quantitative, analyses feasible. While TDA rests on strong mathematical foundations from topology, in applications, it has faced challenges due to difficulties in handling issues of statistical reliability and robustness, often leading to an inability to make scientific claims with verifiable levels of statistical confidence. We propose a methodology for the parametric representation, estimation, and replication of persistence diagrams, the main diagnostic tool of TDA. The power of the methodology lies in the fact that even if only one persistence diagram is available for analysis-the typical case for big data applications-the replications permit conventional statistical hypothesis testing. The methodology is conceptually simple and computationally practical, and provides a broadly effective statistical framework for persistence diagram TDA analysis. We demonstrate the basic ideas on a toy example, and the power of the parametric approach to TDA modeling in an analysis of cosmic microwave background (CMB) nonhomogeneity.

Advances in Measurement Invariance and Mean Comparison of Latent Variables: Equivalence Testing and A Projection-Based Approach.

Measurement invariance (MI) entails that measurements in different groups are comparable, and is a logical prerequisite when studying difference or change across groups. MI is commonly evaluated using multi-group structural equation modeling through a sequence of chi-square and chi-square-difference tests. However, under the conventional null hypothesis testing (NHT) one can never be confident enough to claim MI even when all test statistics are not significant. Equivalence testing (ET) has been recently proposed to replace NHT for studying MI. ET informs researchers a size of possible misspecification and allows them to claim that measurements are practically equivalent across groups if the size of misspecification is smaller than a tolerable value. Another recent advancement in studying MI is a projection-based method under which testing the cross-group equality of means of latent traits does not require the intercepts equal across groups. The purpose of this article is to introduce the key ideas of the two advancements in MI and present a newly developed R package equaltestMI for researchers to easily apply the two methods. A real data example is provided to illustrate the use of the package. It is advocated that researchers should always consider using the two methods whenever MI needs to be examined.

Redhyte: a self-diagnosing, self-correcting, and helpful hypothesis analysis platform

ABSTRACTAn interactive platform for ‘Rapid exploration of data and hypothesis testing’, named Redhyte, is described in this article. Redhyte provides a more efficient and encompassing hypothesis testing procedure than the conventional statistical hypothesis testing framework, by integrating the latter with data-mining techniques. Redhyte is self-diagnosing (it tries to detect whether the user is doing a valid statistical test), self-correcting (it tries to propose and make corrections to the user’s statistical test), and helpful (it searches for promising or interesting hypotheses related to the initial user-specified hypothesis). Hypothesis mining in Redhyte consists of the following steps: context mining, mined-hypothesis formulation, mined-hypothesis scoring on interestingness, and statistical adjustments. And Redhyte supports multiple hypothesis-mining metrics (e.g. several forms of difference lift) that are useful for capturing and evaluating specific aspects of interestingness (e.g. changes in trends and manner of shrinkage). Redhyte is implemented as an R shiny web application and can be found online at https://tohweizhong.shinyapps.io/redhyte, and the source codes can be found at https://github.com/tohweizhong/redhyte.

Design of the myechild trial, an international randomised phase III clinical trial in children with acute myeloid leukaemia incorporating an embedded dose finding study

Acute Myeloid Leukaemia is a rare disease in children but is a significant cause of childhood cancer mortality. This makes running effective clinical trials in the area very difficult. This trial will test strategies in both induction and consolidation for their value in improving survival by evaluating treatments in four randomised comparisons. The embedded dose finding study (EDFS) aims to identify the optimum dose of Gemtuzumab ozogamicin that can be safely combined with induction chemotherapy, which will then form part of the induction randomised comparisons. A rolling design is applied to the EDFS to reduce the need to pause recruitment between cohorts. Patient's treatment pathways will be determined by their cytogenetics, molecular characteristics and morphological response. 700 patients will be randomised over 6 years. While randomisations 2 and 4 are based on conventional hypothesis testing, randomisation 1 and 3 will use a probability based approach to assess event free survival and relapse free survival. This involves plotting bayesian posterior probability distributions using non-informative priors and observed data to infer the probability that one treatment is better than the other. 280 events are anticipated in randomisation 1, if the observed hazard ratio was 0.89 or better in favour of a particular treatment we could be > 80% sure that this is the more effective treatment. Each randomisation will be analysed in their own right and where appropriate be stratified by previous randomisations. No interaction between treatments is anticipated. This design will allow effective evaluation of multiple randomised comparisons in small patient populations.

Assessing Structural Equation Models by Equivalence Testing With Adjusted Fit Indexes

Conventional null hypothesis testing (NHT) is a very important tool if the ultimate goal is to find a difference or to reject a model. However, the purpose of structural equation modeling (SEM) is to identify a model and use it to account for the relationship among substantive variables. With the setup of NHT, a nonsignificant test statistic does not necessarily imply that the model is correctly specified or the size of misspecification is properly controlled. To overcome this problem, this article proposes to replace NHT by equivalence testing, the goal of which is to endorse a model under a null hypothesis rather than to reject it. Differences and similarities between equivalence testing and NHT are discussed, and new “T-size” terminology is introduced to convey the goodness of the current model under equivalence testing. Adjusted cutoff values of root mean square error of approximation (RMSEA) and comparative fit index (CFI) corresponding to those conventionally used in the literature are obtained to facilitate the understanding of T-size RMSEA and CFI. The single most notable property of equivalence testing is that it allows a researcher to confidently claim that the size of misspecification in the current model is below the T-size RMSEA or CFI, which gives SEM a desirable property to be a scientific methodology. R code for conducting equivalence testing is provided in an appendix.

Exchanging the liquidity hypothesis: Delay discounting of money and self-relevant non-money rewards

Evidence that primary rewards (e.g., food and drugs of abuse) are discounted more than money is frequently attributed to money’s high degree of liquidity, or exchangeability for many commodities. The present study provides some evidence against this liquidity hypothesis by contrasting delay discounting of monetary rewards (liquid) and non-monetary commodities (non-liquid) that are self-relevant and utility-matched. Ninety-seven (97) undergraduate students initially completed a conventional binary-choice delay discounting of money task. Participants returned one week later and completed a self-relevant commodity delay discounting task. Both conventional hypothesis testing and more-conservative tests of statistical equivalence revealed correspondence in rate of delay discounting of money and self-relevant commodities, and in one magnitude condition, less discounting for the latter. The present results indicate that liquidity of money cannot fully account for the lower rate of delay discounting compared to non-money rewards.

1402 Design of the Myechild trial, an international randomised phase III clinical trial in children with acute myeloid leukaemia incorporating an embedded dose finding study

Acute Myeloid Leukaemia is a rare disease in children but is a significant cause of childhood cancer mortality. This makes running effective clinical trials in the area very difficult. This trial will test strategies in both induction and consolidation for their value in improving survival by evaluating treatments in four randomised comparisons. The embedded dose finding study (EDFS) aims to identify the optimum dose of Gemtuzumab ozogamicin that can be safely combined with induction chemotherapy, which will then form part of the induction randomised comparisons. A rolling design is applied to the EDFS to reduce the need to pause recruitment between cohorts. Patient’s treatment pathways will be determined by their cytogenetics, molecular characteristics and morphological response. 700 patients will be randomised over 6 years. While randomisations 2 and 4 are based on conventional hypothesis testing, randomisation 1 and 3 will use a probability based approach to assess event free survival and relapse free survival. This involves plotting bayesian posterior probability distributions using non-informative priors and observed data to infer the probability that one treatment is better than the other. 280 events are anticipated in randomisation 1, if the observed hazard ratio was 0.89 or better in favour of a particular treatment we could be > 80% sure that this is the more effective treatment. Each randomisation will be analysed in their own right and where appropriate be stratified by previous randomisations. No interaction between treatments is anticipated. This design will allow effective evaluation of multiple randomised comparisons in small patient populations.

Detecting Structure in Activity Sequences

Can humans discriminate whether strings of events (e.g., shooting success in basketball) were generated by a random or constrained process (e.g., hot and cold streaks)? Conventional wisdom suggests that humans are not good at this discrimination. Following from Cooper, Hammack, Lemasters, and Flach (2014), a series of Monte Carlo simulations and an empirical experiment examined the abilities of both humans and statistical tests (Wald-Wolfowitz Runs Test and 1/f) to detect specific constraints that are representative of plausible factors that might influence the performance of athletes (e.g., learning, non-stationary task constraints). Using a performance/success dependent learning constraint that was calibrated to reflect shooting percentages representative of shooting in NBA games, we found that the conventional null hypothesis tests were unable to detect this constraint as being significantly different from random. Interestingly however, the analysis of human performance showed that people were able to make this discrimination reliably better than chance. Hence, people may also be able to detect patterned/constrained processes in a real-world setting (e.g., streaks in basketball performance), thus supporting the belief in the hot hand.

Supporting Exploratory Hypothesis Testing and Analysis

Conventional hypothesis testing is carried out in a hypothesis-driven manner. A scientist must first formulate a hypothesis based on what he or she sees and then devise a variety of experiments to test it. Given the rapid growth of data, it has become virtually impossible for a person to manually inspect all data to find all of the interesting hypotheses for testing. In this article, we propose and develop a data-driven framework for automatic hypothesis testing and analysis. We define a hypothesis as a comparison between two or more subpopulations. We find subpopulations for comparison using frequent pattern mining techniques and then pair them up for statistical hypothesis testing. We also generate additional information for further analysis of the hypotheses that are deemed significant. The number of hypotheses generated can be very large, and many of them are very similar. We develop algorithms to remove redundant hypotheses and present a succinct set of significant hypotheses to users. We conducted a set of experiments to show the efficiency and effectiveness of the proposed algorithms. The results show that our system can help users (1) identify significant hypotheses efficiently, (2) isolate the reasons behind significant hypotheses efficiently, and (3) find confounding factors that form Simpson’s paradoxes with discovered significant hypotheses.

Bayesian Approaches to Imputation, Hypothesis Testing, and Parameter Estimation

This chapter introduces three applications of Bayesian inference to common and novel issues in second language research. After a review of the critiques of conventional hypothesis testing, our focus centers on ways Bayesian inference can be used for dealing with missing data, for testing theory‐driven substantive hypotheses without a default null hypothesis, and for extending the findings of meta‐analyses to Bayesian estimations of parameters hypothesized to be larger or smaller than those derived from earlier research summaries. Missing data is examined by taking a complete data set and decimating it to simulate a missing at random data set, one common in longitudinal research. Data imputation is then applied to replace the missing data. Comparisons of the full and replaced data sets are made to show that valid inferences about change are possible after imputation of missing data has taken place. The second application of Bayesian inference is demonstrated with the use of a confirmatory analysis of variance approach in which theory‐driven ordered prior hypotheses about mean differences are postulated before the means are compared. Posterior model probabilities and Bayes Factors are discussed as criteria for supporting or refuting the hypothesized mean differences. The third application of Bayesian analysis takes the results of a validity generalization meta‐analysis of aptitude for foreign language learning and tests if a median parameter estimate derived from the validity generalization will be corroborated against data from a language categorized as difficult to learn.

Gaussian Mixture Models and Model Selection for [18F] Fluorodeoxyglucose Positron Emission Tomography Classification in Alzheimer's Disease.

We present a method to discover discriminative brain metabolism patterns in [18F] fluorodeoxyglucose positron emission tomography (PET) scans, facilitating the clinical diagnosis of Alzheimer’s disease. In the work, the term “pattern” stands for a certain brain region that characterizes a target group of patients and can be used for a classification as well as interpretation purposes. Thus, it can be understood as a so-called “region of interest (ROI)”. In the literature, an ROI is often found by a given brain atlas that defines a number of brain regions, which corresponds to an anatomical approach. The present work introduces a semi-data-driven approach that is based on learning the characteristics of the given data, given some prior anatomical knowledge. A Gaussian Mixture Model (GMM) and model selection are combined to return a clustering of voxels that may serve for the definition of ROIs. Experiments on both an in-house dataset and data of the Alzheimer’s Disease Neuroimaging Initiative (ADNI) suggest that the proposed approach arrives at a better diagnosis than a merely anatomical approach or conventional statistical hypothesis testing.

Scheme for opportunistic spectrum access in cognitive radio

In this study, an opportunistic spectrum access model with learning strategy is presented for cognitive radio system. Consider a hidden Markov model in learning process, where the ternary hypothesis testing scheme is proposed to perform sensing with the goodness of fit testing. By using a gradient method, the secondary user can estimate the channel patterns and keep up with the variations of the primary user activities. An opportunistic access channel capacity is introduced to evaluate the quality of service of the objective licensed channel. Also, a partially observable Markov decision process framework is presented to exploit spectrum holes. Further analysis shows that, unlike the binary hypothesis testing where the idle state is always protected, the idea of the proposed ternary hypothesis testing puts both the idle and busy state in the same position, which reflects the real state of the licensed channel more precisely. Simulation results indicate that the proposed ternary hypothesis testing scheme outperforms the conventional binary hypothesis testing for both the goodness of fit testing and the energy detection.

Detecting Potential Safety Issues in Large Clinical or Observational Trials by Bayesian Screening When Event Counts Arise from Poisson Distributions

Patients in large clinical trials and in studies employing large observational databases report many different adverse events, most of which will not have been anticipated at the outset. Conventional hypothesis testing of between group differences for each adverse event can be problematic: Lack of significance does not mean lack of risk, the tests usually are not adjusted for multiplicity, and the data determine which hypotheses are tested. This article describes a Bayesian screening approach that does not test hypotheses, is self-adjusting for multiplicity, provides a direct assessment of the likelihood of no material drug–event association, and quantifies the strength of the observed association. The criteria for assessing drug-event associations can be determined by clinical or regulatory considerations. In contrast to conventional approaches, the diagnostic properties of this new approach can be evaluated analytically. Application of the method to findings from a vaccine trial yields results similar to those found by methods using a false discovery rate argument or a hierarchical Bayes approach. [Supplemental materials are available for this article. Go to the publisher's online edition of Journal of Biopharmaceutical Statistics for the following free supplemental resource: Appendix R: Code for calculations.]

Ternary Hypothesis Testing Scheme for Rapidly Opportunistic Spectrum Access

This letter presents a partially observable Markov decision process framework for the opportunistic spectrum access model, where a ternary hypothesis testing scheme is proposed to perform sensing with the goodness of fit testing. Except for idle and busy states, the proposed ternary hypothesis testing also considers the transition state, which reflects the real state of a licensed channel more precisely. Simulation results indicate that the proposed ternary hypothesis testing scheme achieves better performances in channel utilization and collision rate compared with those of the conventional binary hypothesis testing scheme, especially for the rapid variations of the primary user activities.

Posterior Association Networks and Functional Modules Inferred from Rich Phenotypes of Gene Perturbations

Combinatorial gene perturbations provide rich information for a systematic exploration of genetic interactions. Despite successful applications to bacteria and yeast, the scalability of this approach remains a major challenge for higher organisms such as humans. Here, we report a novel experimental and computational framework to efficiently address this challenge by limiting the ‘search space’ for important genetic interactions. We propose to integrate rich phenotypes of multiple single gene perturbations to robustly predict functional modules, which can subsequently be subjected to further experimental investigations such as combinatorial gene silencing. We present posterior association networks (PANs) to predict functional interactions between genes estimated using a Bayesian mixture modelling approach. The major advantage of this approach over conventional hypothesis tests is that prior knowledge can be incorporated to enhance predictive power. We demonstrate in a simulation study and on biological data, that integrating complementary information greatly improves prediction accuracy. To search for significant modules, we perform hierarchical clustering with multiscale bootstrap resampling. We demonstrate the power of the proposed methodologies in applications to Ewing's sarcoma and human adult stem cells using publicly available and custom generated data, respectively. In the former application, we identify a gene module including many confirmed and highly promising therapeutic targets. Genes in the module are also significantly overrepresented in signalling pathways that are known to be critical for proliferation of Ewing's sarcoma cells. In the latter application, we predict a functional network of chromatin factors controlling epidermal stem cell fate. Further examinations using ChIP-seq, ChIP-qPCR and RT-qPCR reveal that the basis of their genetic interactions may arise from transcriptional cross regulation. A Bioconductor package implementing PAN is freely available online at http://bioconductor.org/packages/release/bioc/html/PANR.html.

Hypothesis Testing in Noninferiority and Equivalence MRMC ROC Studies

Conventional multireader multicase receiver operating characteristic (MRMC ROC) methodologies use hypothesis testing to test differences in diagnostic accuracies among several imaging modalities. The general MRMC-ROC analysis framework is designed to show that one modality is statistically different among a set of competing modalities (ie, the superiority setting). In practice, one may wish to show that the diagnostic accuracy of a modality is noninferior or equivalent, in a statistical sense, to that of another modality instead of showing its superiority (a higher bar). The purpose of this article is to investigate the appropriate adjustments to the conventional MRMC ROC hypothesis testing methodology for the design and analysis of noninferiority and equivalence hypothesis tests. We present three methodological adjustments to the updated and unified Obuchowski-Rockette (OR)/Dorfman-Berbaum-Metz (DBM) MRMC ROC method for use in statistical noninferiority/equivalence testing: 1) the appropriate statement of the null and alternative hypotheses; 2) a method for analyzing the experimental data; and 3) a method for sizing MRMC noninferiority/equivalence studies. We provide a clinical example to further illustrate the analysis of and sizing/power calculation for noninferiority MRMC ROC studies and give some insights on the interplay of effect size, noninferiority margin parameter, and sample sizes. We provide detailed analysis and sizing computation procedures for a noninferiority MRMC ROC study using our method adjusted from the updated and unified OR/DBM MRMC method. Likewise, we show that an equivalence hypothesis test is identical to performing two simultaneous noninferiority tests (ie, either modality is noninferior to the other). Conventional MRMC ROC methodology developed for superiority studies can and should be adjusted appropriately for the design and analysis of a noninferiority/equivalence hypothesis testing. In addition, the confidence interval of the difference in diagnostic accuracies is important information and should generally accompany the statistical analysis and any conclusions drawn from the hypothesis testing.

Target detection with randomized thresholds for lidar applications

Light detection and ranging (lidar) systems use binary hypothesis tests to detect the presence of a target in a range interval. For systems that count photon detections, hypothesis test thresholds are normally set so that a target detection is declared if the number of detections exceeds a particular number. When this method is employed, the false alarm probability can not be selected arbitrarily. In this paper, a hypothesis test that uses randomized thresholds is described. This randomized method of thresholding allows lidar operation at any false alarm probability. When there is a maximum allowable false alarm probability, the hypothesis test that uses randomized thresholds generally produces higher target detection probabilities than the conventional (nonrandom) hypothesis test.