Constructing group sequential tests with data monitoring

Abstract

Objective:Group sequential tests are procedures for repeatedly analysing accruing data, and aim to reduce test time by stopping data collection as soon as an effect is deemed statistically present or absent. Data monitoring, on the other hand, aims to control statistical power by repeatedly re-estimating sample variance from the accruing data, and then adjusting the sample size accordingly. The current work presents a numerical framework for combining group sequential tests with data monitoring, thus leveraging the best of both methods. Methods:The approach revolves around numerically convolving truncated probability density functions, which leads to numerically tracktable distributions for the sequential test statistic. This then allows conventional power analysis methodologies to be used during the data monitoring procedure, thus keeping the procedure relatively intuitive and transparent. The test operating characteristics of the procedure were evaluated extensively in simulations and in Auditory Brainstem Response data. Results:A good control over statistical power was observed, which was attributed to data monitoring. In some test conditions, however, data monitoring led to inflated type-I error rates and a reduced control over statistical power. This was attributed to sample variance estimation errors, and was largely overcome by initiating data monitoring after at least ∼40-90 samples had accrued and/or by leaving data blinded. Conclusion/Significance:When there is uncertainty regarding the effect size and/or the population variance, then group sequential tests with data monitoring may offer a solution, providing an efficient means to reach an unambiguous test outcome in terms of effect present/absent.