Evaluation of HackLearn COFELET Game User Experience for Cybersecurity Education

Abstract

HackLearn is a scenario-based hacking simulation game for teaching cybersecurity concepts while providing hands-on hacking experiences to the learners. HackLearn design is based on the COFELET framework, which assimilates modern learning theories, well-known cybersecurity standards, and built-in scaffolding and assessment features. Aiming at evaluating the user experience perceived by HackLearn’s users, we describe the process of adopting it in a real educational environment based on the didactic framework for simulation games. Additionally, we present the evaluation methodology elaborated, based on the serious games’ quality characteristics framework. We discuss the evaluation results which indicate that HackLearn is engaging, motivating, usable and effective in teaching cybersecurity concepts and hacking strategies and techniques. The evaluation results revealed the HackLearn’s aspects that can be improved such as the scaffolding feature and the communication mechanism with the game’s back-end facility. The presented work validates and finalizes prior work elaborated on the COFELET framework (e.g., COFELET ontology and the COFELET games life-cycle), whereas it provides directions for future work in the development and evaluation of cybersecurity serious games.