Multiple Hypothesis Testing Procedures Research Articles

A general framework for statistical performance comparison of evolutionary computation algorithms

This paper proposes a statistical methodology for comparing the performance of evolutionary computation algorithms. A twofold sampling scheme for collecting performance data is introduced, and these data are analyzed using bootstrap-based multiple hypothesis testing procedures. The proposed method is sufficiently flexible to allow the researcher to choose how performance is measured, does not rely upon distributional assumptions, and can be extended to analyze many other randomized numeric optimization routines. As a result, this approach offers a convenient, flexible, and reliable technique for comparing algorithms in a wide variety of applications.

An Adaptive Single‐step FDR Procedure with Applications to DNA Microarray Analysis

The use of multiple hypothesis testing procedures has been receiving a lot of attention recently by statisticians in DNA microarray analysis. The traditional FWER controlling procedures are not very useful in this situation since the experiments are exploratory by nature and researchers are more interested in controlling the rate of false positives rather than controlling the probability of making a single erroneous decision. This has led to increased use of FDR (False Discovery Rate) controlling procedures. Genovese and Wasserman proposed a single-step FDR procedure that is an asymptotic approximation to the original Benjamini and Hochberg stepwise procedure. In this paper, we modify the Genovese-Wasserman procedure to force the FDR control closer to the level alpha in the independence setting. Assuming that the data comes from a mixture of two normals, we also propose to make this procedure adaptive by first estimating the parameters using the EM algorithm and then using these estimated parameters into the above modification of the Genovese-Wasserman procedure. We compare this procedure with the original Benjamini-Hochberg and the SAM thresholding procedures. The FDR control and other properties of this adaptive procedure are verified numerically.

Resampling‐based multiple hypothesis testing procedures for genetic case‐control association studies

In case-control studies of unrelated subjects, gene-based hypothesis tests consider whether any tested feature in a candidate gene--single nucleotide polymorphisms (SNPs), haplotypes, or both--are associated with disease. Standard statistical tests are available that control the false-positive rate at the nominal level over all polymorphisms considered. However, more powerful tests can be constructed that use permutation resampling to account for correlations between polymorphisms and test statistics. A key question is whether the gain in power is large enough to justify the computational burden. We compared the computationally simple Simes Global Test to the min P test, which considers the permutation distribution of the minimum p-value from marginal tests of each SNP. In simulation studies incorporating empirical haplotype structures in 15 genes, the min P test controlled the type I error, and was modestly more powerful than the Simes test, by 2.1 percentage points on average. When disease susceptibility was conferred by a haplotype, the min P test sometimes, but not always, under-performed haplotype analysis. A resampling-based omnibus test combining the min P and haplotype frequency test controlled the type I error, and closely tracked the more powerful of the two component tests. This test achieved consistent gains in power (5.7 percentage points on average), compared to a simple Bonferroni test of Simes and haplotype analysis. Using data from the Shanghai Biliary Tract Cancer Study, the advantages of the newly proposed omnibus test were apparent in a population-based study of bile duct cancer and polymorphisms in the prostaglandin-endoperoxide synthase 2 (PTGS2) gene.

Detecting periodic patterns in unevenly spaced gene expression time series using Lomb–Scargle periodograms

Periodic patterns in time series resulting from biological experiments are of great interest. The commonly used Fast Fourier Transform (FFT) algorithm is applicable only when data are evenly spaced and when no values are missing, which is not always the case in high-throughput measurements. The choice of statistic to evaluate the significance of the periodic patterns for unevenly spaced gene expression time series has not been well substantiated. The Lomb-Scargle periodogram approach is used to search time series of gene expression to quantify the periodic behavior of every gene represented on the DNA array. The Lomb-Scargle periodogram analysis provides a direct method to treat missing values and unevenly spaced time points. We propose the combination of a Lomb-Scargle test statistic for periodicity and a multiple hypothesis testing procedure with controlled false discovery rate to detect significant periodic gene expression patterns. We analyzed the Plasmodium falciparum gene expression dataset. In the Quality Control Dataset of 5080 expression patterns, we found 4112 periodic probes. In addition, we identified 243 probes with periodic expression in the Complete Dataset, which could not be examined in the original study by the FFT analysis due to an excessive number of missing values. While most periodic genes had a period of 48 h, some had a period close to 24 h. Our approach should be applicable for detection and quantification of periodic patterns in any unevenly spaced gene expression time-series data.

Selection of a Time-Varying Quadratic Volterra Model Using a Wavelet Packet Basis Expansion

We consider the identification of a time-varying nonlinear system based on a single realization of the system input-output. To enable identification, the system's time variation is approximated by a weighted sum of known basis sequences. Using wavelet packet basis sequences increases the flexibility of the model, allowing a suitable basis to be selected. A basis selection procedure is formulated using the Best Basis algorithm to choose the minimum entropy wavelet packet basis. The statistical significance of each of the chosen basis sequences is then tested using a multiple hypothesis testing procedure. Selecting individual sequences in this way achieves a specified level of confidence that the final model contains only those sequences that are significant. As an alternative, we also propose a search procedure, based on an information theoretic criterion, to select basis sequences.

A CFAR Design for a Window Spanning Two Clutter Fields

When the heterogeneous clutter field spanning the spatial sampling sliding window can be modeled as two contiguous homogeneous clutter fields with the statistical parameters of each field unknown and independent from field to field and with the transition point between fields also not known, then the cell-averaging constant false alarm rate (CFAR) performance significantly degrades, yielding target masking effects and loss of false alarm regulation. For the same defined and encountered environment spanning the sliding window, the performance degradation effects are shown to be largely eliminated when a newly developed class of CFAR tests is employed. These tests are designated as heterogeneous clutter estimating CFARs (HCE-CFAR). The test initially involves the combined use of multiple hypothesis testing and maximum likelihood estimation procedures to estimate the statistical parameters of each of the two fields, and the transition point between them, and then makes use of the relevant estimated clutter field parameters to effect the final decision rule. HCE-CFAR designs are presented for both the cases when the contiguous fields have Rayleigh first-order probability distributions, and log-normal probability distribution. However, the focus of the development and the conducted performance evaluation is for the Rayleigh clutter cases.