The Effect of Stopping Rules on the Evaluation of Audit Evidence

Abstract

Does the value of audit evidence depend on whether a fixed sample was taken, or sampling was stopped when some specified outcome was achieved? This study extends the literature on probabilistic inference in auditing by empirically examining the relevance of stopping rules to judgment in planning work on internal controls. Statisticians debate whether stopping rules should have inferential value: Bayesians say not, but frequentists may conclude differently about a hypothesis depending on whether a sample size was determined in advance or an optional stopping rule was used. In a 2×2×2×2 mixed between- and within-subjects experiment, subjects assess the strength of evidence obtained using different stopping rules. Stopping rule is manipulated between subjects; strength of evidence, direction of evidence, and sampler are manipulated within subjects. A linear mixed model is estimated with restricted maximum likelihood. We find that stopping rules strongly influence subjects' judgment: audit evidence obtained using a sampling plan that allows for optional early stopping is attributed less weight than evidence obtained using a plan that fixes the sample size in advance. This is consistent with arguments in cognitive psychology that human statistical intuition conforms better to the frequentist than to the Bayesian paradigm of inference (Gigerenzer 1994; Cosmides and Tooby 1996). We find no support for the hypothesis that the reduction in perceived strength of evidence between a fixed and an optional stopping rule will be greater if the sampling is carried out by someone else; we find that evidence strength and direction also affect audit judgment.