Nonconservative exact small-sample inference for discrete data

Abstract

Exact small-sample methods for discrete data use probability distributions that do not depend on unknown parameters. However, they are conservative inferentially: the actual error probabilities for tests and confidence intervals are bounded above by the nominal level. This article surveys ways of reducing or even eliminating the conservatism. Fuzzy inference is a recent innovation that enables one to achieve the error probability exactly. We present a simple way of conducting fuzzy inference for discrete one-parameter exponential family distributions. In practice, most scientists would find this approach unsuitable yet might be disappointed by the conservatism of ordinary exact methods. Thus, we recommend using exact small-sample distributions but with inferences based on the mid-P value. This approach can be motivated by fuzzy inference, it is less conservative than standard exact methods, yet usually it does well in terms of achieving desired error probabilities. We illustrate for inferences about the binomial parameter.