What Aspects Do Non-Functional Requirements in App User Reviews Describe? An Exploratory and Comparative Study

Abstract

App user reviews have been regarded as a valuable source to elicit user requirements. Existing research on eliciting requirements from user reviews mainly focuses on functional aspects. However, Non-Functional Requirements (NFRs) from diverse sources play a critical role during software development. In this paper, we conducted an exploratory and comparative study in order to gain a deeper understanding on the nature of NFRs in user reviews, and to further compare the difference on the distributions of various NFRs between user reviews and industrial requirements specifications with respect to the system properties that NFRs address. We used 1278 user review sentences from two popular Apps (iBooks and WhatsApp) classified as NFRs that address quality characteristics, and further classified these 1278 NFRs based on the system view and the behavior theory and compared the difference on the distributions of various NFRs between user reviews and industrial requirements specifications. The results show that in App user reviews, users primarily report quality aspects on Reliability and Usability, and over 50% NFRs address interface behavior of the systems. Overall the distributions of NFRs with respect to the system view and the behavior theory in user reviews and industrial requirements specifications are similar, but the distributions of NFRs classified as architecture and state, and the distributions of NFRs in certain quality characteristic classes (e.g., the Usability class) show some differences. We concluded that most NFRs in user reviews are essentially not non-functional since they describe behavior over the interface of the systems. Compared with NFRs in industrial requirements specifications, NFRs in user reviews report more state-related aspects of the systems and care less about the architecture of the systems.