A Comparative Evaluation of Belief Revision Models in Auditing

Abstract

External auditors typically gather audit evidence in a sequential fashion and revise their estimates of likelihood of material misstatements based on the evidence collected. Optimal utilization of audit evidence can help control audit risk and improve audit efficiency and effectiveness. This paper first shows how a typical audit risk assessment and belief revision task can be modeled using four theoretical models of belief revision. Then the descriptive properties of these models are evaluated based on the actual judgments of experienced auditors who assessed the likelihood of error in a task involving inventory valuation. A realistic audit case was administered to one hundred and one experienced auditors. Models based on the following theories were evaluated: a version of Bayesian inference labeled Cascaded Inference Theory (Schum 1987; Schum & DuCharme 1971), two versions of the Belief Adjustment Model (Hogarth & Einhorn 1992), and a version of the Dempster-Shafer Theory of Belief Functions (Srivastava and Shafer 1992; Shafer 1976). The experimental task involves belief revision after combining evidence about the reliability of a client's internal control system with substantive evidence relating to inventory pricing. Both the experimental and manipulation check results show that the auditors were sensitive to differences in the source reliability and diagnosticity of the manipulated audit evidence. The analytical results reveal that the four models differ in the way they interpret evidence relating to internal control system reliability and the manner in which such evidence is aggregated with price test evidence. Yet, all four models correctly predict the direction of auditor belief revision. The study reveals that important structural differences in the models result in differences in the magnitude, but not the direction of belief revision. This theoretical and empirical evidence has heretofore been unavailable in the literature. Further, the version of the Hogarth and Einhorn Belief Adjustment Model that views control systems reliability as negative evidence is the only model that captures both the direction and magnitude of auditors' belief revision. Finally, auditors' belief revision was lower than that predicted by the remaining three models, with the extent of discounting ranging from 31% to 40%. Under-utilization of evidential value with respect to any of the models implies opportunities to improve audit performance either through training or with decision aids. The paper discusses implications for both audit theory and practice.