Aesthetics and Power Considerations in Multiple Testing – A Contradiction?

Abstract

In this paper we discuss aesthetical concepts and requirements for reasonable multiple test procedures. Aesthetical considerations lead to logical decision patterns which are conceivable and, if possible, simple to use and to communicate. Such considerations are sometimes contradictory to the ubiquitous requirement of maximizing power for a multiple test procedure. We illustrate the necessary trade-offs with several examples. We start by considering important logical properties and then discuss three different concepts of monotonicity. Afterwards we have a closer look at the recently proposed "fallback procedure" and show that it has some less appealing properties. Finally, we investigate the distribution of the numbers of significant results with respect to both expectation and variance.