Software developers need help too! Developing a methodology to analyse cognitive dimension-based feedback on usability

Abstract

ABSTRACT Software developers use various methods to evaluate usability and identify usability issues that exist in systems they develop. Cognitive dimensions framework (CDF) based usability evaluation is one of the popular usability evaluation methods. It uses an open-ended questionnaire to collect qualitative feedback from users after using a system. To identify usability issues, evaluators should analyse this qualitative feedback. However, the approach to follow when performing this analysis is not explored in detail. We conducted a systematic literature review and reviewed 70 studies that used various CDF questionnaires for usability evaluations and investigated how those studies have analysed CDF questionnaire responses to identify usability issues. This revealed five methods that previous research has used for data analysis and four methods for identifying usability issues from CDF questionnaire responses. We applied the results of the literature review to develop a methodology and a set of guidelines to analyse qualitative feedback collected via a CDF questionnaire that targets evaluating security application programming interfaces. We tested the developed guidelines by conducting an empirical investigation. The results of the experiment revealed that using the proposed guidelines helps to identify significantly more usability issues with a higher validity.