Measuring Team Knowledge During Skill Acquisition of a Complex Task

Abstract

The design of team training programs and other team interventions could benefit from an understanding of team cognition. The research presented in this article evaluates methods for eliciting and assessing team knowledge during acquisition of a complex task. Knowledge measures are evaluated in terms of their ability to predict team performance and also in terms of how they reflect skill acquisition. The study was conducted in the context of a synthetic 3-person team task that involved operating an uninhabited air vehicle (UAV). Eleven teams of 3 Air Force ROTC cadets participated in 3 experimental sessions lasting from 3 to 6 hr. During these sessions, teams were trained on the task and were observed as they performed ten 40-min missions. During the missions, team performance and team process behaviors were measured, as well as the fleeting team knowledge associated with situation awareness. In addition, long-term team knowledge regarding both taskwork and teamwork were measured offline in four sessions. Results indicated that teams reached asymptotic performance on this task after 1.5 hr of individual training and four 40-min team missions. This skill acquisition was paralleled by improvements in team situation models, teamwork knowledge, and, to a lesser extent, team process behaviors. Taskwork relatedness ratings measured at both the individual and team level were good predictors of team performance and indicated that high performing teams had more knowledge of the task from the perspective of other team members, as opposed to lower performing teams. These measures help reveal the knowledge underlying team behavior and thus have implications for team training and other interventions.