Using cognitive dimensions to evaluate the usability of security APIs: An empirical investigation

Abstract

Abstract Context Usability issues of security Application Programming Interfaces (APIs) are a main factor for mistakes programmers make that could result in introducing security vulnerabilities into applications they develop. This has become a common problem as there is no methodology to evaluate the usability of security APIs. A usability evaluation methodology for security APIs would allow API developers to identify usability issues of security APIs and fix them. A Cognitive Dimensions Framework (CDF) based usability evaluation methodology has been proposed in previous research to empirically evaluate the usability of security APIs. Objective In this research, we evaluated the proposed CDF based methodology through four security APIs (Google Authentication API, Bouncy Castle light weight Crypto API, Java Secure Socket Extension API, OWASP Enterprise Security API). Method We conducted four experiments where in each experiment we recruited programmers and they completed a programming task using one of the four security APIs. Participants’ feedback on each cognitive dimension of the particular API was collected using the cognitive dimensions questionnaire. Usability issues of each API was identified based on this feedback. Results Results of the four experiments revealed that over 83% of the usability issues in a security API could be identified by this methodology with a considerably good validity and reliability. Conclusion The proposed CDF based usability evaluation methodology provides a good platform to conduct usability evaluation for security APIs.