Abstract
Recently, mobile devices are more popular than computers. However, mobile apps are not as thoroughly tested as desktop ones, especially for graphical user interface (GUI). In this paper, we study the detection and segmentation of graphical elements on GUIs for mobile apps based on deep learning. It is the preliminary work of GUI testing for mobile apps based on artificial intelligence. We create a dataset, which consists of 2,100 GUI screenshots (or pages) labeled with 42,156 graphic elements in 8 classes. Based on our dataset, we adopt Mask R-CNN to train the detection and segmentation of graphic elements on GUI screenshots. The experimental results show that the mAP value achieves 98%.
Published Version
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call