Developing and validating a serious game‐based assessment for cyber occupations in the US Air Force

Abstract

AbstractWe present initial structural validity evidence for a serious game designed for personnel selection and classification for cybersecurity roles in the US Air Force (USAF). Based on literature review and input from USAF cybersecurity subject‐matter‐experts, we targeted six constructs for assessment. We describe the development process used to build a game to assess individual differences in these constructs, while also being engaging and motivating for players. We attend to the challenge of avoiding an overall game performance factor that dominates variance of multiple constructs scored from the same gameplay episodes and report steps taken to enhance discriminant validity of the scores. We apply factor analysis and item response theory models to develop scores that are reliable, show discriminant validity, and show modest education/gender group differences.