Psychological validity of uncertainty combining rules in expert systems

Abstract

Abstract: In recent years, an increasing amount of research in Artificial Intelligence has focused on the representation and manipulation of uncertainty in expert systems. New approaches have been developed and justified because of their mathematical rigor or simplicity, cautiousness, programmability, and/or ability to capture the essence of natural language. Little attention has been paid to the psychological validity of any approach, new or old. To explore the psychological validity of rules that combine uncertainties, Oak Ridge National Laboratory (ORNL) has developed a computer program known as LES. Four applications of LES revealed that subjects’ natural combining rules are (1) not best modeled by any generally accepted uncertainty approach and (2) may be highly idiosyncratic. Knowledge Engineers should be aware that cognitive representations of knowledge in expert systems are not routinely being matched with combining rules actually used by experts. Frustrating development activities and highly unsatisfactory expert system performance may result.