Reliability and validity of the Cybersecurity Awareness INventory (CAIN)

Abstract

ABSTRACT The Cybersecurity Awareness INventory (CAIN) was introduced by Tempestini and colleagues (2023, “The Cybersecurity Awareness INventory (CAIN): Early Phases of Development of a Tool for Assessing Cybersecurity Knowledge Based on the ISO/IEC 27032.” Journal of Cybersecurity and Privacy 3 (1): 61–75) as a proficiency test designed to estimate individuals’ knowledge of cyber threats. The questionnaire was based on ISO/IEC 27032:2012 (Information Technology – Security Techniques – Guidelines for Cybersecurity) and consisted of 46 items. In this paper, we present the results of a survey study conducted on a sample of 1511 Italian workers belonging to the same large organisation. The administration of the CAIN to an Italian sample forced us to make an adaptation to the Italian language of the questionnaire. Using Rasch analysis, we improved the CAIN by eliminating some items that did not fit the expected response patterns, leading to a newer and more parsimonious 30-item version. the CAIN score was also used as the dependent variable in a series of ANOVA designs using several security behaviours as factors. That allowed us to test its predictive validity, confirming the ability of the instrument to discriminate between different user profiles with respect to reported security behaviours. Overall, the revised test showed better performance in terms of reliability and validity.