Employing user profiles to test a new version of a GUI component in its context of use

Abstract

Advancement in reusable component technology has had a significant impact on the development of complex graphical user interfaces (GUIs), which are front-ends to most of today's software. Software developers can, with very little effort, integrate components into their software's GUI. Problems, however, arise when new versions of GUI components replace their predecessors in an implementation. Often, the inclusion of a new version of a component breaks some part of the software, i.e., tasks that end-users were able to perform before modifications were made can no longer be performed. Software developers (who also happen to be component users) are unable to perform adequate regression testing in this context because of several factors, including lack of source code, environmental side-effects on GUI rendering, event-driven nature of GUIs, and large number of possible permutations of events. This paper presents a new technique that end-users' quality assurance (QA) teams can employ to test the new version of a component in its application context by using the existing version as a baseline. The technique combines lightweight event-level dynamic profiling to collect user profiles in a transparent manner, GUI reverse engineering to extract the structure of the component's GUI, test case execution to replay the collected profiles on the new version, and GUI oracles that collect properties from the existing version. Empirical studies demonstrate the practicality, usefulness, and limitations of the technique.