Sociometrics and observational assessment of teaming and leadership in a cyber security defense competition

Abstract

Advancing our understanding about the human dynamics of cyber security is a major research challenge. At this point, it is unclear how cyber defense teams are organized and led in coordinating and working together to mount and conduct an effective cyber defense. Therefore, we do not know what makes a cyber defense team more or less effective in responding to and mitigating cyber attacks. Cyber competitions offer an approach to train and evaluate the performance of cyber defense teams; such competitions are now regularly conducted at the high school, college, professional, and military defense levels. These naturalistic exercises of teamwork for cyber defense represent an important source for understanding the way defense teams form, coordinate and organize, and to determine the factors that make teams more or less successful. For this purpose, we participated in data collection at the Mid-Atlantic Collegiate Cyber Defense Competition (MACCDC) to understand the key features of effective team processes defined by outcome measures of scored team success. We collected data from wearable social sensors to assess face-to-face interactions and using a 16-point teamwork instrument called OAT (Observational Assessment of Teamwork) to assess teamwork and leadership behaviors in cyber defense. Importantly, this being a cyber defense competition, the success of these teams is evaluated along three independent scoring dimensions: (a) Maintaining Services, (b) Incidence Response, and (c) Scenario Injects. Our results indicate that the leadership dimension and face-to-face interactions are important factors that determine the success of these teams. Teams with effective leadership were more successful, and face-to-face interactions emerged as a strong negative predictor of success. Thus, functional specialization within a team and well-guided leadership could be important predictors of timely detection and mitigation of ongoing cyber attacks. Future research should address more concrete aspects of team coordination and evaluation in cyber defense teams.