Semantic Similarity-Based Mobile Application Isomorphic Graphical User Interface Identification

Abstract

Applying robots to mobile application testing is an emerging approach to automated black-box testing. The key to supporting automated robot testing is the efficient modeling of GUI elements. Since the application under testing often contains a large number of similar GUIs, the GUI model obtained often contains many redundant nodes. This causes the state space explosion of GUI models which has a serious effect on the efficiency of GUI testing. Hence, how to accurately identify isomorphic GUIs and construct quasi-concise GUI models are key challenges faced today. We thus propose a semantic similarity-based approach to identifying isomorphic GUIs for mobile applications. Using this approach, the information of GUI elements is first identified by deep learning network models, then, the GUI structure model feature vector and the semantic model feature vector are extracted and finally merged to generate a GUI embedding vector with semantic information. Finally, the isomorphic GUIs are identified by cosine similarity. Then, three experiments are conducted to verify the generalizability and effectiveness of the method. The experiments demonstrate that the proposed method can accurately identify isomorphic GUIs and shows high compatibility in terms of cross-platform and cross-device applications.