Reporting Usability Defects: A Systematic Literature Review

Abstract

Usability defects can be found either by formal usability evaluation methods or indirectly during system testing or usage. No matter how they are discovered, these defects must be tracked and reported. However, empirical studies indicate that usability defects are often not clearly and fully described. This study aims to identify the state of the art in reporting of usability defects in the software engineering and usability engineering literature. We conducted a systematic literature review of usability defect reporting drawing from both the usability and software engineering literature from January 2000 until March 2016. As a result, a total of 57 studies were identified, in which we classified the studies into three categories: reporting usability defect information, analysing usability defect data and key challenges. Out of these, 20 were software engineering studies and 37 were usability studies. The results of this systematic literature review show that usability defect reporting processes suffer from a number of limitations, including: mixed data, inconsistency of terms and values of usability defect data, and insufficient attributes to classify usability defects. We make a number of recommendations to improve usability defect reporting and management in software engineering.