Fisher Randomization Test Research Articles

On exact randomization-based covariate-adjusted confidence intervals.

Randomization-based inference using the Fisher randomization test allows for the computation of Fisher-exact P-values, making it an attractive option for the analysis of small, randomized experiments with non-normal outcomes. Two common test statistics used to perform Fisher randomization tests are the difference-in-means between the treatment and control groups and the covariate-adjusted version of the difference-in-means using analysis of covariance. Modern computing allows for fast computation of the Fisher-exact P-value, but confidence intervals have typically been obtained by inverting the Fisher randomization test over a range of possible effect sizes. The test inversion procedure is computationally expensive, limiting the usage of randomization-based inference in applied work. A recent paper by Zhu and Liu developed a closed form expression for the randomization-based confidence interval using the difference-in-means statistic. We develop an important extension of Zhu and Liu to obtain a closed form expression for the randomization-based covariate-adjusted confidence interval and give practitioners a sufficiency condition that can be checked using observed data and that guarantees that these confidence intervals have correct coverage. Simulations show that our procedure generates randomization-based covariate-adjusted confidence intervals that are robust to non-normality and that can be calculated in nearly the same time as it takes to calculate the Fisher-exact P-value, thus removing the computational barrier to performing randomization-based inference when adjusting for covariates. We also demonstrate our method on a re-analysis of phase I clinical trial data.

A Randomization-Based Theory for Preliminary Testing of Covariate Balance in Controlled Trials

Randomized trials balance all covariates on average and are the gold standard for estimating treatment effects. Chance imbalances nevertheless exist more or less in realized treatment allocations and intrigue an important question: what should we do if the treatment groups differ with respect to some important baseline characteristics? A common strategy is to conduct a preliminary test of the balance of baseline covariates after randomization, and invoke covariate adjustment for subsequent inference if and only if the realized allocation fails some prespecified criterion. Although such practice is intuitive and popular among practitioners, the existing literature has so far only evaluated its properties under strong parametric model assumptions in theory and simulation, yielding results of limited generality. To fill this gap, we examine two strategies for conducting preliminary test-based covariate adjustment by regression, and evaluate the validity and efficiency of the resulting inferences from the randomization-based perspective. The main result is 2-fold. First, the preliminary-test estimator based on the analysis of covariance can be even less efficient than the unadjusted difference in means, and risks anticonservative confidence intervals based on normal approximation even with the robust standard error. Second, the preliminary-test estimator based on the fully interacted specification is less efficient than its counterpart under the always-adjust strategy, and yields overconservative confidence intervals based on normal approximation. In addition, although the Fisher randomization test is still finite-sample exact for testing the sharp null hypothesis of no treatment effect on any individual, it is no longer valid for testing the weak null hypothesis of zero average treatment effect in large samples even with properly studentized test statistics. These undesirable properties are due to the asymptotic non-normality of the preliminary-test estimators. Based on theory and simulation, we echo the existing literature and do not recommend the preliminary-test procedure for covariate adjustment in randomized trials.

Ligand and Structure-Based Pharmacophore Modelling, Computer-aided Screening and Molecular Docking to Detect Novel NS5B Polymerase Inhibitors as Anti-HCV Compounds

The ongoing interest of researchers in the direct-acting NS5B inhibitors in the development of viral disease hepatitis has attracted our attention in the direction of the development of a quantitative four-featured pharmacophore model containing HBA (2), HY (1), and PI (1) features. The model showed correlation coefficient, RMSD, and cost difference values as 0.895, 0.911, and 30.896, respectively. The model validation is done by using Fisher's randomization test (99%), internal and external tests set the expectation with r2 values of 0.80 and 0.65. Simultaneously, the 3D crystal structure of NS5B was utilized for generating a pharmacophore model design based on a structure with features generation 2 Hydrogen Bond Acceptors, 2 Hydrogen Bond Donors, and 2 Hydrophilic features. Further on these two models, HITS searches using NCI and Maybridge databases were done. Three hundred forty-five compounds were screened, and the three greatest potent hits were docked to the active site of NS5B. It was found that these docked conformations showed interactions with GLN446 and TYR448 amino acids, which are located at NS5B active site. In an instant, with the use of various computational methods in a sequence, we have identified novel structurally diverse NS5B inhibitors.

Leveraging the Fisher Randomization Test using Confidence Distributions: Inference, Combination and Fusion Learning

AbstractThe flexibility and wide applicability of the Fisher randomization test (FRT) make it an attractive tool for assessment of causal effects of interventions from modern-day randomized experiments that are increasing in size and complexity. This paper provides a theoretical inferential framework for FRT by establishing its connection with confidence distributions. Such a connection leads to development’s of (i) an unambiguous procedure for inversion of FRTs to generate confidence intervals with guaranteed coverage, (ii) new insights on the effect of size of the Monte Carlo sample on the estimation of a p-value curve and (iii) generic and specific methods to combine FRTs from multiple independent experiments with theoretical guarantees. Our developments pertain to finite sample settings but have direct extensions to large samples. Simulations and a case example demonstrate the benefit of these new developments.

Randomization Tests for Weak Null Hypotheses in Randomized Experiments

The Fisher randomization test (FRT) is appropriate for any test statistic, under a sharp null hypothesis that can recover all missing potential outcomes. However, it is often sought after to test a weak null hypothesis that the treatment does not affect the units on average. To use the FRT for a weak null hypothesis, we must address two issues. First, we need to impute the missing potential outcomes although the weak null hypothesis cannot determine all of them. Second, we need to choose a proper test statistic. For a general weak null hypothesis, we propose an approach to imputing missing potential outcomes under a compatible sharp null hypothesis. Building on this imputation scheme, we advocate a studentized statistic. The resulting FRT has multiple desirable features. First, it is model-free. Second, it is finite-sample exact under the sharp null hypothesis that we use to impute the potential outcomes. Third, it conservatively controls large-sample Type I error under the weak null hypothesis of interest. Therefore, our FRT is agnostic to the treatment effect heterogeneity. We establish a unified theory for general factorial experiments and extend it to stratified and clustered experiments. Supplementary materials for this article are available online.

Investigations of Structural Requirements for BRD4 Inhibitors through Ligand- and Structure-Based 3D QSAR Approaches.

The bromodomain containing protein 4 (BRD4) recognizes acetylated histone proteins and plays numerous roles in the progression of a wide range of cancers, due to which it is under intense investigation as a novel anti-cancer drug target. In the present study, we performed three-dimensional quantitative structure activity relationship (3D-QSAR) molecular modeling on a series of 60 inhibitors of BRD4 protein using ligand- and structure-based alignment and different partial charges assignment methods by employing comparative molecular field analysis (CoMFA) and comparative molecular similarity indices analysis (CoMSIA) approaches. The developed models were validated using various statistical methods, including non-cross validated correlation coefficient (r2), leave-one-out (LOO) cross validated correlation coefficient (q2), bootstrapping, and Fisher’s randomization test. The highly reliable and predictive CoMFA (q2 = 0.569, r2 = 0.979) and CoMSIA (q2 = 0.500, r2 = 0.982) models were obtained from a structure-based 3D-QSAR approach using Merck molecular force field (MMFF94). The best models demonstrate that electrostatic and steric fields play an important role in the biological activities of these compounds. Hence, based on the contour maps information, new compounds were designed, and their binding modes were elucidated in BRD4 protein’s active site. Further, the activities and physicochemical properties of the designed molecules were also predicted using the best 3D-QSAR models. We believe that predicted models will help us to understand the structural requirements of BRD4 protein inhibitors that belong to quinolinone and quinazolinone classes for the designing of better active compounds.

A randomization-based perspective on analysis of variance: a test statistic robust to treatment effect heterogeneity

Summary Fisher randomization tests for Neyman’s null hypothesis of no average treatment effect are considered in a finite-population setting associated with completely randomized experiments involving more than two treatments. The consequences of using the $F$ statistic to conduct such a test are examined, and we argue that under treatment effect heterogeneity, use of the $F$ statistic in the Fisher randomization test can severely inflate the Type I error under Neyman’s null hypothesis. We propose to use an alternative test statistic, derive its asymptotic distributions under Fisher’s and Neyman’s null hypotheses, and demonstrate its advantages through simulations.

The statistical study of quasi‐periodic oscillations of the radio emission in solar quiet regions

AbstractIn this study we discuss variations of the radio emission from the Quiet Sun Areas (QSA) at centimeter wavelength (1.76 cm). Data were obtained from Nobeyama Radioheliograph (NoRH). Oscillations of selected areas were studied carefully from data taken over one week. We try to find quasi‐periodic solar oscillations from the QSA. We used the traditional Fast Fourier Transform (FFT), Global Wavelet Spectrum (GWS) and Wavelet (Morlet) for studying signals in the frequency/time‐frequency domain. We used the Fisher randomization test to verify the significance of the observed signal. Instrumental and sky noises were studied using a cross‐correlation analysis. Additionally, a single pixel analysis were done. Wide ranges of solar oscillation periods were found from the Quiet Sun Area (QSA): 3–15, 35–70, and 90 minutes. Some physical explanations are suggested for these oscillations. However, it is not possible to give a conclusive statement about the origin of the long quasi‐periodic (>60 min) oscillations from the QSA (© 2011 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Synthesis, Biological Evaluation, and Three-Dimensional in Silico Pharmacophore Model for σ1 Receptor Ligands Based on a Series of Substituted Benzo[d]oxazol-2(3H)-one Derivatives

Novel benzo[d]oxazol-2(3H)-one derivatives were designed and synthesized, and their affinities against sigma receptors were evaluated. On the basis of 31 compounds, a three-dimensional pharmacophore model for the sigma(1) receptor binding site was developed using the Catalyst 4.9 software package. The best 3D pharmacophore hypothesis, consisting of one positive ionizable, one hydrogen bond acceptor, two hydrophobic aromatic, and one hydrophobic features provided a 3D-QSAR model with a correlation coefficient of 0.89. The best hypothesis was also validated by three independent methods, i.e., the Fisher randomization test included in the CatScramble functionality of Catalyst, the leave-one-out test, and activity prediction of an additional test set. The achieved results will allow researchers to use this 3D pharmacophore model for the design and synthesis of a second generation of high affinity sigma(1) ligands, as well as to discover other lead compounds for this class of receptors.

Three-dimensional quantitative structure-activity relationship analysis of a set of Plasmodium falciparum dihydrofolate reductase inhibitors using a pharmacophore generation approach.

A 3D pharmacophore model able to quantitatively predict inhibition constants was derived for a series of inhibitors of Plasmodium falciparum dihydrofolate reductase (PfDHFR), a validated target for antimalarial therapy. The data set included 52 inhibitors, with 23 of these comprising the training set and 29 an external test set. The activity range, expressed as Ki, of the training set molecules was from 0.3 to 11 300 nM. The 3D pharmacophore, generated with the HypoGen module of Catalyst 4.7, consisted of two hydrogen bond donors, one positive ionizable feature, one hydrophobic aliphatic feature, and one hydrophobic aromatic feature and provided a 3D-QSAR model with a correlation coefficient of 0.954. Importantly, the type and spatial location of the chemical features encoded in the pharmacophore were in full agreement with the key binding interactions of PfDHFR inhibitors as previously established by molecular modeling and crystallography of enzyme-inhibitor complexes. The model was validated using several techniques, namely, Fisher's randomization test using CatScramble, leave-one-out test to ensure that the QSAR model is not strictly dependent on one particular compound of the training set, and activity prediction in an external test set of compounds. In addition, the pharmacophore was able to correctly classify as active and inactive the dihydrofolate reductase and aldose reductase inhibitors extracted from the MDDR database, respectively. This test was performed in order to challenge the predictive ability of the pharmacophore with two classes of inhibitors that target very different binding sites. Molecular diversity of the data sets was finally estimated by means of the Tanimoto approach. The results obtained provide confidence for the utility of the pharmacophore in the virtual screening of libraries and databases of compounds to discover novel PfDHFR inhibitors.

Power comparison of non-parametric tests: Small-sample properties from Monte Carlo experiments

Summary Non-parametric tests that deal with two samples include scores tests (such as the Wilcoxon rank sum test, normal scores test, logistic scores test, Cauchy scores test, etc.) and Fisher's randomization test. Because the non-parametric tests generally require a large amount of computational work, there are few studies on small-sample properties, although asymptotic properties with regard to various aspects were studied in the past. In this paper, the non-parametric tests are compared with the t -test through Monte Carlo experiments. Also, we consider testing structural changes as an application in economics.

The Rank Sum Test with Confidence Limits

Click to increase image sizeClick to decrease image sizeConfidence LimitsNormal ApproximationRank Sum Test Additional informationNotes on contributorsLloyd S. NelsonSum test and as the Mann-Whitney test. Actually, it was developed independently by a number of workers as reported in a historical survey by Kruskal (1957). To perform the test, observations are replaced by their size ranks and the ranks are then analyzed by what amounts to Fisher's randomization test, which is discussed by Conover (1980, pp. 328–330).

Exact Calculation of Permutational Distributions for Two Dependent Samples I

AbstractA simple Fortran subroutine is given for the calculation of permutational distributions. Important special cases are Fisher's randomization test, the Wilcoxon signed ranks test and the sign test. The algorithm works in polynomial time. Thus it can be used even for micro‐computers within justifiable time limits.

Refinements in Scent-Station Methodology for Assessing Trends in Carnivore Populations

Refinements in Scent-Station Methodology for Assessing Trends in Carnivore Populations

Randomization Analysis of Experimental Data: The Fisher Randomization Test

Abstract R.A. Fisher's classic text on the design of experiments is the principal source of inspiration for a mode of data interpretation that is usually characterized as randomization analysis. In Chapter III of this text, Fisher briefly commented on how to make a randomization test on some data generated by a Darwin experiment. Two variants of this randomization test are discussed in this article. The variant that is discussed in Section 4 may be regarded as the forerunner of all nonparametric tests. The original variant of the test is discussed in Section 6. The author concludes that the Fisher randomization test is not logically viable.

Randomization and statistical inference

An expository discussion is presented of the arguments concerning the role of randomization in statistical inference from the results of a randomized experiment. Then some comparisons of the power of the Fisher randomization test to the normal theory F-test for the case of an unpaired, two-group, randomized experiment are given..

The behaviour of some significance tests under experimental randomization

For the past 40 years, there has been confusion on the differences between tests of significance and tests of hypotheses to the point where data interpretation is presented as an accept-reject process. This paper discusses the case of inference for a simple experiment, the paired design. It gives a rationale of the significance tester, and proceeds to a comparison of three tests of significance which can be viewed as competitors in the case of this design. These are the sign test, S, the Wilcoxon test, W, and the Fisher randomization test, R. The conclusion is that there is indeed an ordering of the tests, which is R preferred to W preferred to S. The S test should never be used if the others are possible. An Appendix gives a proof of monotonicity of the R test with respect to a shift alternative.