Usability evaluation of a comprehensive national health information system: A heuristic evaluation

Abstract

Abstract Aim to evaluate the usability of a comprehensive national health information system by the heuristic method. Introduction Presently, information systems are widely being used in healthcare settings. Methods Five independent evaluators assessed the user interface design of this system in terms of its compliance with a set of predetermined standard principles, also known as Jakob Nielsen's 10 general principles. Problems were reassessed in the presence of all evaluators, and similar cases were merged and a single list of unique problems was prepared. After a second assessment, the evaluators determined and categorized problem severity in five domains, including: the absence of a problem (zero point), a cosmetic problem (1 point), a minor problem (2 points), a major problem (3 points) and a catastrophic problem (4 points). Data were then analyzed in a spreadsheet using descriptive statistical tests. Results The “recognition rather than recall” principle with 13 problems (21.3% of all cases) had the greatest frequency among all problems, while the “match between system and the real world” and “help and documentation” principles with 1 problem (1.6% of all cases) had the least frequency. Moreover, principles such as “help users recognize, diagnose and recover from errors”, “error prevention”, and “help and documentation” had a mean severity of 2.8, 2.8, and 3.4, respectively. Consequently, they were considered as catastrophic and major problems. Conclusions Based on the viewpoint of evaluation experts, a large portion of problems in this system were classified into major and catastrophic categories, which primarily indicates the poor usability of this system. Therefore, it is highly recommended that authorities be notified of the issues in writing in order to resolve them in a future update. Finally, special consideration should be given to the meticulous evaluation of these systems during preliminary stages of design and development, so as to encounter fewer issues on a national level at the time of implementation.