Permutation tests for ASCA in multivariate longitudinal intervention studies

Abstract

AbstractPermutation tests are the standard technique for significance testing in Analysis of Variance Simultaneous Component Analysis. However, there is a vast number of alternative approaches for permutation testing, and the number of choices grows in relation to the complexity of the study design. In this paper, we focus on longitudinal intervention studies with multivariate outcomes, a relevant experimental design in clinical studies where the outcome is an omics profile (such as in genomics, metabolomics, and the like). We propose a new technique to derive power curves tailored to the size and (un)balanced nature of the data set in the study. This technique is useful to identify misleading permutation tests, with lack of power or overly optimistic outcomes. We found that choosing the best permutation approach is far from intuitive and that there is a significant risk of deriving incorrect conclusions in real‐life analyses. Our approach avoids this risk and can be extended to other complex designs of interest. The code is available for free use.