Performance Of Statistical Tests Research Articles

Empirical power comparison of statistical tests in contemporary phase III randomized controlled trials with time-to-event outcomes in oncology

More than 95% of recent cancer randomized controlled trials used the log-rank test to detect a treatment difference making it the predominant tool for comparing two survival functions. As with other tests, the log-rank test has both advantages and disadvantages. One advantage is that it offers the highest power against proportional hazards differences, which may be a major reason why alternative methods have rarely been employed in practice. The performance of statistical tests has traditionally been investigated both theoretically and numerically for several patterns of difference between two survival functions. However, to the best of our knowledge, there has been no attempt to compare the performance of various statistical tests using empirical data from past oncology randomized controlled trials. So, it is unknown whether the log-rank test offers a meaningful power advantage over alternative testing methods in contemporary cancer randomized controlled trials. Focusing on recently reported phase III cancer randomized controlled trials, we assessed whether the log-rank test gave meaningfully greater power when compared with five alternative testing methods: generalized Wilcoxon, test based on maximum of test statistics from multiple weighted log-rank tests, difference in t-year event rate, and difference in restricted mean survival time with fixed and adaptive . Using manuscripts from cancer randomized controlled trials recently published in high-tier clinical journals, we reconstructed patient-level data for overall survival (69 trials) and progression-free survival (54 trials). For each trial endpoint, we estimated the empirical power of each test. Empirical power was measured as the proportion of trials for which a test would have identified a significant result (p value < .05). For overall survival, t-year event rate offered the lowest (30.4%) empirical power and restricted mean survival time with fixed offered the highest (43.5%). The empirical power of the other types of tests was almost identical (36.2%-37.7%). For progression-free survival, the tests we investigated offered numerically equivalent empirical power (55.6%-61.1%). No single test consistently outperformed any other test. The empirical power assessment with the past cancer randomized controlled trials provided new insights on the performance of statistical tests. Although the log-rank test has been used in almost all trials, our study suggests that the log-rank test is not the only option from an empirical power perspective. Near universal use of the log-rank test is not supported by a meaningful difference in empirical power. Clinical trial investigators could consider alternative methods, beyond the log-rank test, for their primary analysis when designing a cancer randomized controlled trial. Factors other than power (e.g. interpretability of the estimated treatment effect) should garner greater consideration when selecting statistical tests for cancer randomized controlled trials.

Comparison of Statistical Data Models for Identifying Differentially Expressed Genes Using a Generalized Likelihood Ratio Test

Currently, statistical techniques for analysis of microarray-generated data sets have deficiencies due to limited understanding of errors inherent in the data. A generalized likelihood ratio (GLR) test based on an error model has been recently proposed to identify differentially expressed genes from microarray experiments. However, the use of different error structures under the GLR test has not been evaluated, nor has this method been compared to commonly used statistical tests such as the parametric t-test. The concomitant effects of varying data signal-to-noise ratio and replication number on the performance of statistical tests also remain largely unexplored. In this study, we compared the effects of different underlying statistical error structures on the GLR test's power in identifying differentially expressed genes in microarray data. We evaluated such variants of the GLR test as well as the one sample t-test based on simulated data by means of receiver operating characteristic (ROC) curves. Further, we used bootstrapping of ROC curves to assess statistical significance of differences between the areas under the curves. Our results showed that i) the GLR tests outperformed the t-test for detecting differential gene expression, ii) the identity of the underlying error structure was important in determining the GLR tests' performance, and iii) signal-to-noise ratio was a more important contributor than sample replication in identifying statistically significant differential gene expression.

PHYLOGENETIC EVIDENCE OF A RAPID RADIATION OF PLEUROCARPOUS MOSSES (BRYOPHYTA)

Pleurocarpous mosses, characterized by lateral female gametangia and highly branched, interwoven stems, comprise three orders and some 5000 species, or almost half of all moss diversity. Recent phylogenetic analyses resolve the Ptychomniales as sister to the Hypnales plus Hookeriales. Species richness is highly asymmetric with approximately 100 Ptychomniales, 750 Hookeriales, and 4400 Hypnales. Chloroplast DNA (cpDNA) sequences were obtained to compare partitioning of molecular diversity among the orders with estimates of species richness, and to test the hypothesis that either the Hookeriales or Hypnales underwent a period (or periods) of exceptionally rapid diversification. Levels of biodiversity were quantified using explicitly historical "phylogenetic diversity" and non-historical estimates of standing sequence diversity. Diversification rates were visualized using lineage-through-time (LTT) plots, and statistical tests of alternative diversification models were performed using the methods of Paradis (1997). The effects of incomplete sampling on the shape of LTT plots and performance of statistical tests were investigated using simulated phylogenies with incomplete sampling. Despite a much larger number of accepted species, the Hypnales contain lower levels of (cpDNA) biodiversity than their sister group, the Hookeriales, based on all molecular measures. Simulations confirm previous results that incomplete sampling yields diversification patterns that appear to reflect a decreasing rate through time, even when the true phylogenies were simulated with constant rates. Comparisons between simulated results and empirical data indicate that a constant rate of diversification cannot be rejected for the Hookeriales. The Hypnales, however, appear to have undergone a period of exceptionally rapid diversification for the earliest 20% of their history.

A high performance test of differential gene expression for oligonucleotide arrays

A method, Logit-t, for normalization of microarray data and for finding significantly differentially expressed genes has been described.

Admissibility of unbiased tests for a composite hypothesis with a restricted alternative

This paper discusses α-admissiblility and d-admissiblity which are important concepts in studying the performance of statistical tests for composite hypotheses. A sufficient condition for α-admissibility is presented. When α=1/m, the Nomakuchi-Sakata test, which is uniformly more powerful than the likelihood ratio test for hypotheses min (θ1, θ1) = 0 versus min (θ1, θ1) > 0, is generalized for a class of distributions in an exponential family, and its unbiasedness and α-admissibility are shown. Finally, the case of α≠ 1/m is discussed in brief.