Please please me

Abstract

Mobile application developers have started to realize that quality plays a vital role in increasing the popularity of mobile applications (apps), thereby directly influencing economical profit (in-app purchases revenue) and app-related success factors (i.e., number of downloads). Therefore, developers have become increasingly concerned with taking preemptive actions to ensure the quality of their apps. In general, developers have been relying on testing as their main quality assurance practice. However, little is known about how much mobile app testing contributes to increasing user level satisfaction. In this paper we investigate to what extent testing mobile apps contributes to achieving higher user satisfaction. To this end, we probed into whether there is a relation between having automated tests and overall user satisfaction. We looked into users ratings, which express their level of satisfaction with apps, and users reviews, which often include bug (i.e., fault) reports. By analyzing a quantitative indicator of user satisfaction (i.e., user rating), we found that there is no significant difference between apps with automated tests and apps that have been developed without test suites. We also applied sentiment analysis on user reviews to examine the main differences between apps with and without test suites. The results of our review-based sentiment analysis suggest that most apps with and without test suites score quite high for user satisfaction. In addition, we found that update-related problems are far more common in apps with test suites, while apps without test suites are likely to have battery-drain problems.