The impact of cognitive feedback on judgment performance and trust with decision aids

Abstract

Computerized decision aids are designed to support human operators’ decision-making activities in a variety of contexts including medicine, military command and control, and aviation industry. One common characteristic of these systems is of the role of the decision-aid, integrating a variety of measured information to produce a simple form of more meaningful information that can be used to support human operators’ judgment about environmental states of interest. When these aids malfunction, the decision makers may ignore the aid due to the lack of trust in the aid. This study examines the effect of automated decision aids of varying quality in producing environmental estimates, and investigates the effect of meta-information in supporting judgment performance of human operators’ with the decision aids and calibrating human trust in such aids. A Lens Model based feedback is used to provide meta-information about the decision-aid. An aircraft identification task is performed under varying conditions of aid quality and the presence of meta-information. Results show that performance, as well as assessments of trust in the aid, are affected by the decision aid's quality. More importantly, participants given with the meta-information performed significantly better than those without it. Results indicate that human operators can compensate for a poor performing aid when meta-information is available. Further, operators’ trust was better calibrated corresponding to the decision aid's quality. A practical implication of this study is that meta-information can be useful to human operators in increasing their understanding and appropriate utilization of automated decision aids.