GUI Objects Research Articles

Investigating the robustness of locators in template-based Web application testing using a GUI change classification model

GUI-based test-cases generated by Capture and Replay tools suffer from the well-known fragility problem: they may break even if small layout changes are operated in a Web application, without modifying the app functionality. An approach based on the automatic injection of HTML tag attributes named hooks in the source code of Web templates has been recently proposed to solve this problem. Such hooks allow the unique identification of GUI items to be located during test case execution. This technique showed its effectiveness in a preliminary validation study, where it allowed to significantly reduce the number of test case locator breakages in regression testing of student-made Web applications. This paper presents a further validation study where we compared the robustness of hook-based test cases against state-of-the-art and state-of-the-practice techniques for locating GUI objects. We proposed a three dimensional model for classifying different types of layout changes and used it to define a benchmark of realistic changes. Thanks to the model, we systematically compared the robustness of test cases generated by different techniques with respect to specific types of changes and studied the relationship between fragility issues and types of changes in different test case generation techniques.

GUI Test Script Repair in Regression Testing

GUI testing is a process of testing a software application or functionality of GUI. It is defined as an interface between user and software which provides an easy way to interact with the system. GUI plays an important role in software engineering. GUI-based application requires that a test case consists of sequences of user actions/events to be executed. Selenium is an open-source tool for testing GUI Application by executing test cases whether the GUI Application is working properly or not. In this, we present test script repair technique to repair test cases. Test script repair technique uses reverse engineering process for creating the test script. Test script repair consists of three stages; they are Ripping, Mapping and Repairing. In ripping stage, there are two relationships for representing event interaction of GUI Application. During ripping we know the location of each widget. In mapping stage,original GUI events are mapped to an event-flow graph (EFG). In repairing stage, Event flow graph uses repairing transformations and human input to modified script to repair the test cases, and synthesizes a new “repaired” test script. During this process, test script repair uses GUI objects for yielding a final test script that can be executed using selenium tool to validate the GUI Application. An experiment using selenium tool to test, test cases suggests that it is effective in that unusable test scripts are repaired. Annotations significantly reduced the human cost to repair test cases.

Why does the orientation change mess up my Android application? From GUI failures to code faults

SummaryThis paper investigates the failures exposed in mobile apps by the mobile‐specific event of changing the screen orientation. We focus on GUI failures resulting in unexpected GUI states that should be avoided to improve the apps quality and to ensure better user experience. We propose a classification framework that distinguishes 3 main classes of GUI failures due to orientation changes and exploit it in 2 studies that investigate the impact of such failures in Android apps. The studies involved both open‐source and apps from Google Play that were specifically tested exposing them to orientation change events. The results showed that more than 88% of these apps were affected by GUI failures, some classes of GUI failures were more common than others, and some GUI objects were more frequently involved. The app source code analysis allowed us to identify 6 classes of common faults causing specific GUI failures.

A two-sided collaborative transparent display supporting workspace awareness

Transparent displays naturally support workspace awareness during face-to-face interactions. Viewers see another person’s actions through the display: gestures, gaze, body movements, and what one is manipulating on the display. Yet we can design even better collaborative transparent displays. First, collaborators on either side should be able to directly interact with workspace objects. Second, and more controversially, both sides should be capable of presenting different content. This affords: reversal of images/text in place (so that people on both sides see objects correctly); personal and private territories aligned atop each other; and GUI objects that provide different visuals for feedthrough vs. feedback. Third, the display should visually enhance the gestural actions of the person on the other side to better support workspace awareness. We show how our FacingBoard-2 design supports these collaborative requirements, and confirm via a controlled study that visually enhancing gestures is effective under a range of deteriorating transparency conditions.

To What Extent is Stress Testing of Android TV Applications Automated in Industrial Environments?

An Android-based smart Television (TV) must reliably run its applications in an embedded program environment under diverse hardware resource conditions. Owing to the diverse hardware components used to build numerous TV models, TV simulators are usually not high enough in fidelity to simulate various TV models, and thus are only regarded as unreliable alternatives when stress testing such applications. Therefore, even though stress testing on real TV sets is tedious, it is the de facto approach to ensure the reliability of these applications in the industry. In this paper, we study to what extent stress testing of smart TV applications can be fully automated in the industrial environments. To the best of our knowledge, no previous work has addressed this important question. We summarize the find-ings collected from 10 industrial test engineers to have tested 20 such TV applications in a real production environment. Our study shows that the industry required test automation supports on high-level GUI object controls and status checking, setup of resource conditions and the interplay between the two. With such supports, 87% of the industrial test specifications of one TV model can be fully automated and 71.4% of them were found to be fully reusable to test a subsequent TV model with major up-grades of hardware, operating system and application. It repre-sents a significant improvement with margins of 28% and 38%, respectively, compared to stress testing without such supports.

SITAR: GUI Test Script Repair

System testing of a GUI-based application requires that test cases, consisting of sequences of user actions/events, be executed and the software's output be verified. To enable automated re-testing, such test cases are increasingly being coded as low-level test scripts, to be replayed automatically using test harnesses. Whenever the GUI changes-widgets get moved around, windows get merged-some scripts become unusable because they no longer encode valid input sequences. Moreover, because the software's output may have changed, their test oracles-assertions and checkpoints-encoded in the scripts may no longer correctly check the intended GUI objects. We present ScrIpT repAireR (SITAR), a technique to automatically repair unusable low-level test scripts. SITAR uses reverse engineering techniques to create an abstract test for each script, maps it to an annotated event-flow graph (EFG), uses repairing transformations and human input to repair the test, and synthesizes a new “repaired” test script. During this process, SITAR also repairs the reference to the GUI objects used in the checkpoints yielding a final test script that can be executed automatically to validate the revised software. SITAR amortizes the cost of human intervention across multiple scripts by accumulating the human knowledge as annotations on the EFG. An experiment using QTP test scripts suggests that SITAR is effective in that 41-89 percent unusable test scripts were repaired. Annotations significantly reduced human cost after 20 percent test scripts had been repaired.

GUI objects with impenetrable borders: Instruction (not practice) makes perfect

Walker and Smelcer (Proceedings of the CHI 90, ACM, New York, 1990, pp. 221–225) found that menus could be selected faster if they were placed against the edge of the screen. Doing so creates an impenetrable border between the menu and the edge of the screen that the mouse cursor cannot penetrate. This changes how users move the mouse, so that selection times quicken compared to menus with a penetrable border. Experiment 1 investigated the effect that practice has on the acquisition of objects with and without impenetrable borders. The findings suggest that excessive practice was not necessary to demonstrate border type differences; thus, the advantage of having impenetrable borders seems to be relatively instantaneous. However, it was not readily apparent whether or not participants would realize the advantages of impenetrable borders without instruction. Thus, the primary purpose of Experiment 2 was to assess whether or not users would discover the benefits of impenetrable borders spontaneously. Participants were assigned to either the penetrable or impenetrable border condition. Additionally, participants received either full instruction concerning the benefits of the target placement, or limited instruction that simply informed the participant about the nature of the task. The results demonstrated that participants receiving limited instruction selected targets with impenetrable borders faster than participants who selected objects with a penetrable border. However, an exploratory comparison suggests that only 50% of participants who received limited instruction actually detected the impenetrable border. An additional comparison suggests that with practice the participants who were successful at detecting the impenetrable border selected the targets as quickly as participants who received full instruction concerning the benefits of impenetrable borders. The findings suggest that with full instruction, all users will perform reduced selection times. Given that not all participants discovered the impenetrable border it suggests that, whenever possible, users should receive instruction pertaining to the benefits of the impenetrable borders.

GUI Objects with Impenetrable Borders: Is Instruction Necessary?

Placing menus against the edge of the screen reduces selection times because it creates an impenetrable border between the menu and the edge of the screen that the mouse cursor cannot penetrate (Walker, Smelcer, & Nilsen, 1991). The presence of the impenetrable border changes how users move the mouse, so that selection times quicken compared to menus with a penetrable border. However, it is not readily apparent whether or not GUI users could realize the advantages of impenetrable borders without instruction. The purpose of this experiment was to assess whether or not users would discover the benefits of impenetrable borders spontaneously. Results suggest that only 50% of participants who were unaware of the presence of the impenetrable border actually detected it. Additionally, with practice participants who were successful at detecting the impenetrable border selected the targets as quickly as participants who received full instruction concerning the benefits of impenetrable borders.

The Proactive Desk: A New Haptic Display System for a Digital Desk Using a 2-DOF Linear Induction Motor

The Proactive Desk is a new digital desk with haptic feedback. The concept of a digital desk was proposed by Wellner in 1991 for the first time. A typical digital desk enables a user to seamlessly handle both digital and physical objects on the desk with a common GUI standard. The user, however, handles them as virtual GUI objects. Our Proactive Desk allows the user to handle both digital and physical objects on a digital desk with a realistic feeling. In the Proactive Desk, two linear induction motors are equipped to generate an omnidirectional translational force on the user's hand or on a physical object on the desk without any mechanical links or wires, thereby preserving the advantages of the digital desk. In this article, we first discuss applications of a digital desk with haptic feedback; then we mention the design and structure of the first trial Proactive Desk, and its performance.

A proposal for calling conventions for APL defined functions

This paper will touch on the lack of mechanisms to pass arguments back from defined functions other than return values, if return values are already used as error codes. An extreme case is the use of call-back functions, or event procedures, to the events of GUI objects. In Dyalog APL/W, functionality of arguments and return values of call-back functions are already well-defined. The way you can pass information back from a call-back function, or event procedure, is by storing it as a global variable or by storing it into the Data property of an GUI object, which is also a global variable. This paper proposes a solution - utilize a Call-by-Reference mechanism of function arguments passing, in addition to return values of defined function, to pass results of defined functions back to calling functions. The programming environment is in Dyalog APL/W Version 7.1.3. This interpreter is robust itself and is feature-rich in terms of the integration of APL and MS Windows GUI environment. This paper starts with a description of problems encountered when programming in a GUI environment with call-back functions as well as the difference in design logic of programming styles among procedural programming and event-driven programming. Secondly, this paper presents a brief description of two different mechanisms to pass arguments into and out of defined functions. Thirdly, this paper presents a design solution to the problems mentioned above based on the Call-by-Reference mechanism described in second part. Lastly, a conclusion is made and potential pitfalls are explored.All code segments demonstrated here are implemented and tested in Dyalog APL/W Version 7.1.3.

Writing A Graphical User Interface For Math-Ematica Using Mathematica And MathLink

Interactive graphics are a weak part of Mathematica. In this article, a MathLink program which provides interactive graphics in realtime will be introduced. The program can be used to detect a user interface event, and send it to the Mathematica Kernel as an Event Expression. This mechanism enables one to write Graphical User Inter face (GUI) in Mathematica. To define a GUI object in Mathematica, an Object Oriented Programming Style (OOPS) is introduced. The combination of realtime interaction and kernel evaluation provides a dynamical interface building, execution, and debugging environment. In this article, theoverall design of the program will be discussed.