Risk-Limiting Postelection Audits: Conservative $P$-Values From Common Probability Inequalities

Abstract

Postelection audits of a random sample of batches of ballots against a trustworthy audit trail can limit the risk of certifying an incorrect electoral outcome to alpha, guaranteeing that - if the apparent outcome is wrong - the chance of a full hand count of the audit trail is at least 1-alpha. Risk-limiting audits can be built as sequential tests that audit more batches until either 1) there is strong evidence that the outcome is correct, given the errors found, or 2) there has been a complete hand count. The P -value of the hypothesis that the outcome is wrong is the largest chance, for all scenarios in which the outcome is wrong, that overstatements of the margins between winners and losers would be ldquono largerrdquo than they were observed to be. Different definitions of ldquolargerrdquo give different P -values. A small P -value is strong evidence that the outcome is correct. This paper gives simple approaches to calculating a conservative P-value for several ways of summarizing overstatements and several ways of drawing the sample of batches to audit, emphasizing sampling with probability proportional to a bound up on the error in the pth audit batch (PPEB sampling). A P-value based on Markov's inequality applied to a martingale constructed from the data seems the most efficient among the methods discussed; there are plans to use it to audit contests in two California counties in November 2009.