Assessing multi-agent human-autonomy teams: US Army Robotic Wingman gunnery operations

Abstract

The long-standing body of literature on human teams has provided an important foundation for understanding the core tenets that will structure and govern the dynamic human-autonomy teams of the future. The extant knowledge base highlights the importance of communication as a conduit for the development of teamwork states and processes such as shared situation awareness and appropriate team trust. However, the structure, roles, and interdependencies that will make up these future human-autonomy teams will inherently have marked differences in how they will operate, interact, and function. New metrics and assessment processes are needed to evaluate the interactions of these teams. Included in these needs are metrics of and standards for team performance, changes in team state measures (e.g., performance, stress, team cohesion, and trust), and evaluation of communication as it relates to the effectiveness of the team as a whole. In order to fully develop these metrics and assessments, coordination outside the laboratory environment with real-world autonomy-enabled systems is a critical next step. This paper provides a review of current processes recommended for assessing and evaluating teams, as well as the current limitations. The review is followed by a use case from a field study with the US Army Robotic Wingman program. This use case provides additional options for assessment protocols for human-autonomy team performance, team communication, and human state (including behavioral and physiological indicators) while operating in a real-world setting during gunnery operations. Overall, outcomes provide insight into future assessment procedures for future team crew station interaction with multiple robotic assets.