Inferring Independent Sets of Gaussian Variables after Thresholding Correlations

Abstract

We consider testing whether a set of Gaussian variables, selected from the data, is independent of the remaining variables. This set is selected via a very simple approach: these are the variables for which the correlation with all other variables falls below some threshold. Unlike other settings in selective inference, failure to account for the selection step leads to excessively conservative (as opposed to anti-conservative) results. We propose a new test that conditions on the event that the selection resulted in the set of variables in question, and thus is not overly conservative. To achieve computational tractability, we develop a characterization of the conditioning event in terms of the canonical correlation between groups of random variables. In simulation studies and in the analysis of gene co-expression networks, we show that our approach has much higher power than a “naive” approach that ignores the effect of selection. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.