Reducing conservatism of exact small-sample methods of inference for discrete data

Abstract

Exact small-sample methods for discrete data use probability distributions that do not depend on unknown parameters. However, they are conservative inferentially: The actual error probabilities for tests and confidence intervals are bounded above by the nominal level. This article discusses ways of reducing the conservatism. Fuzzy inference is a recent innovation that enables one to achieve the error probability exactly. We present a simple way of conducting fuzzy inference for discrete one-parameter exponential family distributions. In practice, most scientists would find this approach unsuitable yet might be disappointed by the conservatism of ordinary exact methods. Thus, to use exact small-sample distributions, we recommend inferences based on the mid-P value. This approach can be motivated by fuzzy inference, it is less conservative than standard exact methods, yet usually it does well in terms of achieving desired error probabilities. We illustrate this and other small-sample methods for the case of inferences about the binomial parameter.