Comparing Levels and Types of Situational-Awareness based Agent Transparency in Human-Agent Collaboration

Abstract

Increasing agent transparency is an ongoing challenge for Human-Agent Collaboration (HAC). Chen et al. proposed the three level SAT framework to improve Agent Transparency and users’ Situational Awareness (SA) by informing about (1) what the agent is doing, (2) why the agent is doing it and (3) what the agent will do next. Explanations can be descriptive (informing the user decision-making process) or prescriptive (guiding the user toward a pre-determined choice). To study these differences, we conducted a 3 (SA level) x 2 (explanation types) online between-group user experiment (n=180) where we designed six visual explanations and tested their impact on task performance, reliance, reported trust, cognitive load and situational awareness in a goal-oriented HAC interactive task. We found that SA level 1 explanations led to better task performance, while SA level 2 explanations increased trust. Moreover, descriptive explanations had a more positive impact on participants compared to prescriptive explanations.