Heuristics for evaluating IT security management tools

Pooya Jaferian,Konstantin Beznosov,Kirstie Hawkey,Andreas Sotirakopoulos,Maria Velez-Rojas

doi:10.1145/2078827.2078837

Pooya Jaferian, Konstantin Beznosov + Show 3 more

Open Access

https://doi.org/10.1145/2078827.2078837

Copy DOI

Publication Date: Jul 20, 2011
Citations: 14	License type: cc-by

Affiliation: University of British Columbia, Dalhousie University

Abstract

The usability of IT security management (ITSM) tools is hard to evaluate by regular methods, making heuristic evaluation attractive. However, standard usability heuristics are hard to apply as IT security management occurs within a complex and collaborative context that involves diverse stakeholders. We propose a set of ITSM usability heuristics that are based on activity theory, are supported by prior research, and consider the complex and cooperative nature of security management. In a between-subjects study, we compared the employment of the ITSM and Nielsen's heuristics for evaluation of a commercial identity management system. Participants who used the ITSM set found more problems categorized as severe than those who used Nielsen's. As evaluators identified different types of problems with the two sets of heuristics, we recommend employing both the ITSM and Nielsen's heuristics during evaluation of ITSM tools.

Highlights

Information technology security management (ITSM) tools serve several purposes including protection, detection, and user management
Our results show that the evaluation of the Identity Management (IdM) system requires more evaluators compared to evaluations performed by Nielsen on simple user interfaces; we observed few overlaps between problems identified by individual evaluators using either Nielsen’s or the IT security management (ITSM) heuristics
We presented heuristics for the evaluation of ITSM tools

Summary

Introduction

Information technology security management (ITSM) tools serve several purposes including protection (e.g., network, system, and data), detection (e.g., tools for threat and vulnerability management), and user management Werlinger et al [57] identified nine security activities that require collaborative interactions and developed a model of the complexity of their interactions This complexity arises from organizational attributes (e.g., distribution of IT management); the need for SPs to interact with multiple stakeholders with different perceptions of risk and levels of security training; and their need to engage in multiple security related activities. Each of these activities may require different tacit knowledge and kinds of information to be conveyed. One of the dominant theoretical foundations for HCI has been information processing psychology [24] This theory focuses on human actions as the units of analysis. As prior research shows that social and organizational factors impact ITSM activities, activity theory may be useful when describing the ITSM context

Objectives

Methods

Results

Discussion

Conclusion