Supporting Human-AI Teams:Transparency, explainability, and situation awareness

Abstract

System autonomy and AI are being developed for a wide variety of applications where they will likely work in tandem with people, forming human-AI teams (HAT). Situation awareness (SA) of autonomous systems and AI has been established as critical for effective interaction and oversight of these systems. As AI capabilities grow, and more effective teaming behaviors are expected of AI systems, there will also be an increased need for shared SA between the human and AI teammates. Methods for supporting team SA within HAT are discussed in terms of team SA requirements, team SA mechanisms, team SA displays and team SA processes. A framework for understanding the types of information that needs to be shared within HAT is provided, including a focus on taskwork SA, agent SA, and teamwork SA. AI based on learning systems creates new challenges for the development of good SA and mental models. AI transparency and explainability are discussed in terms of their separate roles for supporting SA and mental models in HAT. The SA Oriented Design (SAOD) process is described as a systematic methodology for developing transparent AI displays for HAT and an example of its application to automated driving in a Tesla is provided.