Combining usability evaluations to highlight the chain that leads from usability flaws to usage problems and then negative outcomes

Abstract

Poor usability of health technology is thought to diminish work system performance, increase error rates and, potentially, harm patients. The present study (i) used a combination of usability evaluation methods to highlight the chain that leads from usability flaws to usage problems experienced by users and, ultimately, to negative patient outcomes, and (ii) validated this approach by studying two different discharge summary production systems. To comply with quality guidelines, the process of drafting and sending discharge summaries is increasingly being automated. However, the usability of these systems may modify their impact (or the absence thereof) in terms of production times and quality, and must therefore be evaluated. Here, we applied three successive techniques for usability evaluation (heuristic evaluation, user testing and field observation) to two discharge summary production systems (underpinned by different technologies). The systems’ main usability flaws led respectively to an increase in the time need to produce a discharge summary and the risk of patient misidentification. Our results are discussed with regard to the possibility of linking the usability flaws, usage problems and the negative outcomes by successively applying three methods for evaluating usability (heuristic evaluation, user testing and in situ observations) throughout the system development life cycle.