A framework for competence development and assessment in hybrid cybersecurity exercises

Abstract

Rising numbers and sophistication of security threats in the digital domain cause an increase in the demand for skilled cybersecurity professionals. In response, cybersecurity exercises, and in particular—cyber defence exercises (CDX) are becoming ever more popular. They provide a training platform to simulate real-life situations. CDX are significant events involving months of preparation, and previous studies show a lack of objective evidence of their relevance regarding the learning impact. Skills of exercise participants are usually different and vary from tech-savvy to beginners. Also, trainees are diverse when considering their background, current work profile (position and institution), and experience. Assessment of their competencies is essential to ensure quality in training. The complexity and multi-dimensionality of the usual CDX make it challenging. Additionally, the costly event usually focuses on just a subset of participants, and non-technical members of an organisation are not included. The goal of our research is to provide a proper methodology to optimise the exercises so that every team and each participant, including a non-technical trainee, are adequately evaluated and trained using the allocated resources most effectively.This paper presents a framework to aid in the development and assessment of cybersecurity competences of all teams during hybrid CDX. The framework aims towards raised cybersecurity awareness—a state when every user of digital technologies understands the associated risks. The framework consists of a sequence of steps including stages of formative assessment, team construction, determination of objectives for different types of teams, and the exercise flow. It complements standard methodologies for cybersecurity training programs. The framework was developed based on data collected using questionnaires, interviews, and direct observation in a case study carried out during international cybersecurity exercises. The framework would help organise hybrid exercises for a diverse community of trainees, including non-technical members of an organisation.