Delta GUI change detection using inferred models

Abstract

Recent software development methodologies emphasize iterative and incremental evolution to align with stakeholders’ needs. This perpetual and rapid software evolution demands ongoing research into verification practices and technologies that ensure swift responsiveness and effective management of software delta increments. Strategies such as code review have been widely adopted for development and verification, ensuring readability and consistency in the delta increments of software projects. However, the integration of techniques to detect and visually report delta changes within the Graphical User Interface (GUI) software applications remains an underutilized process. In this paper, we set out to achieve two objectives. First, we aim to conduct a comprehensive review of existing studies concerning GUI change detection in desktop, web, and mobile applications to recognize common practices. Second, we introduce a novel change detection tool capable of highlighting delta GUI changes for this diverse range of applications. To accomplish our first objective, we performed a systematic mapping of the literature using the Scopus database. To address the second objective, we designed and developed a GUI change detection tool. This tool simultaneously transits and compares state models inferred by a scriptless testing tool, enabling the detection and highlighting of GUI changes to detect the widgets or functionalities that have been added, removed, or modified. Our study reveals the existence of a multitude of techniques for change detection in specific GUI systems with different objectives. However, there is no widely adopted technique suitable for the diverse range of existing desktop, web, and mobile applications. Our tool and findings demonstrate the effectiveness of using inferred state models to highlight between 8 and 20 GUI changes in software delta increments containing a large number of changes over months and between 4 and 6 GUI changes in delta increments of small iterations performed over multiple weeks. Moreover, some of these changes were recognized by the software developers as GUI failures that required a fix. Finally, we expose the motivation for using this technique to help developers and testers analyze GUI changes to validate delta increments and detect potential GUI failures, thereby fostering knowledge dissemination and paving the way to standard practices.