APP constraint analysis approach to select mobile devices for compatibility crowdtesting

Abstract

AbstractThe compatibility issues caused by Android fragmentation have become a vital task in the development of Android applications. To locate those issues, thousand of crowd testers run apps on different devices with different configurations to achieve the largest coverage, which might be costly and time‐consuming. Since existing approaches to selecting optimal devices are device‐side analysis without the information of the internal structures of apps, app‐side analysis that flags the essential devices for testers has remained elusive. To mitigate this gap of compatibility crowdtesting, this paper proposes an app constraint analysis approach named CompatDroid to generate the optimal device set to guide crowd testers. By evaluating 46 benchmark apps on 14 SDK versions, the optimal device sets are successfully generated, and CompatDroid only needs no more than 7 Android versions to achieve almost the same code coverage (i.e., 33.13%) testing on all 14 android versions (i.e., 34.65%) in 36 of 46 apps, which indicates that it can drastically reduce the consumption of test resources while losing little test coverage. On a larger dataset, CompatDroid successfully analyzes 98.3% of 645 apps, in which the median number of the optimal SDK versions set is 2.5 versions, and 68.92% of those apps contain the constraints of SDK version (i.e., SDK version) while 84.86% of them do not have the constraints of hardware information (i.e., model name and manufacture name).