GUI Testing Research Articles

Accounting for defect characteristics in evaluations of testing techniques

As new software-testing techniques are developed, before they can achieve widespread acceptance, their effectiveness at detecting defects must be evaluated. The most common way of evaluating testing techniques is with empirical studies, in which one or more techniques are tried out on software with known defects. However, the defects used can affect the performance of the techniques. To complicate matters, it is not even clear how to effectively describe or characterize defects. To address these problems, this article describes an experiment architecture for empirically evaluating testing techniques which takes both defect and test-suite characteristics into account. As proof of concept, an experiment on GUI-testing techniques is conducted. It provides evidence that the defect characteristics proposed do help explain defect detection, at least for GUI testing, and it explores the relationship between the coverage of defective code and the detection of defects.

XML을 이용한 안드로이드 GUI 위젯의 기능 테스트 자동화

캡처 리플레이 기법은 GUI 테스팅을 위하여 흔히 사용되는 자동화 방법이다. 하지만 안드로이드 플랫폼 애플리케이션 테스팅에는 직접 이용하기 어렵다. 그 이유는 좋은 테스팅 프레임워크가 제공되고 있지만 GUI 요소들과 위젯 이벤트 처리 부분이 자동 링크되지 않기 때문이다. 캡처 리플레이 테스팅 도구가 없다면 명세서를 이용하여 테스팅 시나리오를 만들고 일일이 수동으로 GUI 요소들과 연결하여 테스팅 하여야 한다. 이 논문은 현재 사용되고 있는 안드로이드기반 GUI 자동화 테스트 기법보다 발전되고 최적화된 캡처 리플레이 방법을 제안하고 이의 효율성을 증명하기 위한 것이다. XML을 이용하여 주요 위젯기반 원소를 추출하기 위한 기술들을 정립하고 위젯기반 API 이벤트 처리 방법을 고안하였다. 제안한 방법으로 캡처 단계에 모니터링하여 클릭 이벤트가 일어나는 형태를 추적한 후 리플레이 단계에서 활성화 된 위젯을 API 이벤트와 상태를 교환하여 테스트 케이스를 생성한다. Capture-and-replay technique is a common automatic method for GUI testing. Testing applications on Android platform cannot use directly capture-and-replay technique due to the testing framework which is already set up and technical supported by Google and lack of automatic linking GUI elements to actions handling widget events. Without capture-and-replay testing tools testers must design and implement testing scenarios according to the specification, and make linking every GUI elements to event handling parts all by hand. This paper proposes a more improved and optimized approach than common capture-and-replay technique for automatic testing Android GUI widgets. XML is applied to extract GUI elements from applications based on tracing the actions to handle widget events. After tracing click events using monitoring in capture phase test cases will be created by communicating status of activated widget in replay phase with API events.

UIA 라이브러리를 이용한 GUI 테스트 자동화 방법

When preparing test cases and running the test the existing GUI test tools require many tester`s interventions. To cope with such problem this paper suggests a new method to build test cases for GUI test. This method identifies the potential control flows within the GUI and constructs the GUI map. The UIA library in .NET Framework is used to extract information about the GUI controls and the GUI map is constructed by the extracted information. Test scenarios are generated from the extracted information about the GUI controls using the grouping mechanism. Based on the grouping mechanism, various test scenarios which are test cases in GUI tests can be made by replacing a GUI control by another one in the same group. The existing GUI test tools do not support the concept of test coverage. Since, however, our method survey which part of the GUI map is executed or not during running the test, the test coverage can be measured by using the GUI map.

CTest Integration of Sikuli Automated GUI Testing

In order to test our software GoFigure2 for the visualization and the processing of extremely large 3D+t microscopy images, we utilized the visual programming technology Sikuli to automate our GUI testing. We then integrated these Sikuli tests into CTest and used CDash to report the results on a dashboard with the view to test our software automatically every night and detect bugs as soon as they occur. Our rate of releases more than doubled as a result.

Obstacles and opportunities in deploying model‐based GUI testing of mobile software: a survey

SUMMARYModel‐based testing has not been widely deployed in industry yet. There seem to be both technical and non‐technical reasons for this situation. A survey among mobile software testing professions in Finland was conducted that aimed at investigating possible obstacles and opportunities towards wider deployment of this technology. This paper discusses the results and provides conclusions that indicate that at least in this context, there is much interest among practitioners towards the technology. However, more research is needed to make model creation and maintenance as easy as possible. In addition, metrics should be developed that can be used to report test results in the same manner as using existing testing techniques, enabling comparison between different approaches. Special emphasis should also be placed on enabling quick bug localization. Furthermore, it seems that successful pilot projects are a key to wider industrial adoption. Based on the survey findings, an agenda for the future model‐based testing research is outlined. Copyright © 2011 John Wiley & Sons, Ltd.

Automated GUI performance testing

A significant body of prior work has devised approaches for automating the functional testing of interactive applications. However, little work exists for automatically testing their performance. Performance testing imposes additional requirements upon GUI test automation tools: the tools have to be able to replay complex interactive sessions, and they have to avoid perturbing the application’s performance. We study the feasibility of using five Java GUI capture and replay tools for GUI performance test automation. Besides confirming the severity of the previously known GUI element identification problem, we also describe a related problem, the temporal synchronization problem, which is of increasing importance for GUI applications that use timer-driven activity. We find that most of the tools we study have severe limitations when used for recording and replaying realistic sessions of real-world Java applications and that all of them suffer from the temporal synchronization problem. However, we find that the most reliable tool, Pounder, causes only limited perturbation and thus can be used to automate performance testing. Based on an investigation of Pounder’s approach, we further improve its robustness and reduce its perturbation. Finally, we demonstrate in a set of case studies that the conclusions about perceptible performance drawn from manual tests still hold when using automated tests driven by Pounder. Besides the significance of our findings to GUI performance testing, the results are also relevant to capture and replay-based functional GUI test automation approaches.

Iterative execution-feedback model-directed GUI testing

Current fully automatic model-based test-case generation techniques for GUIs employ a static model. Therefore they are unable to leverage certain state-based relationships between GUI events (e.g., one enables the other, one alters the other’s execution) that are revealed at run-time and non-trivial to infer statically. We present ALT – a new technique to generate GUI test cases in batches. Because of its “alternating” nature, ALT enhances the next batch by using GUI run-time information from the current batch. An empirical study on four fielded GUI-based applications demonstrated that ALT was able to detect new 4- and 5-way GUI interaction faults; in contrast, previous techniques, due to their requirement of too many test cases, were unable to even test 4- and 5-way GUI interactions.

Design and analysis of GUI test-case prioritization using weight-based methods

Testing the correctness of a GUI-based application is more complex than the conventional code-based application. In addition to testing the underlying codes of the GUI application, the space of possible combinations of events with a large GUI-input sequence also requires creating numerous test cases to confirm the adequacy of the GUI testing. Running all GUI test cases and then fixing all found bugs may be time-consuming and delaying the project completion. Hence, it is important to advance the test cases that uncover the most faults as fast as possible in the testing process. Test-case prioritization has been proposed and used in recent years because it can improve the rate of fault detection during the testing phase. However, few studies have discussed the problem of GUI test-case prioritization. In this paper, we propose a weighted-event flow graph for solving the non-weighted GUI test case and ranking GUI test cases based on weight scores. The weighted scores can either be ranked from high to low or be ordered by dynamic adjusted scores. Finally, three experiments are performed, and experimental results show that the adjusted-weight method can obtain a better fault-detection rate.

A Proposal for Automatic Testing of GUIs Based on Annotated Use Cases

This paper presents a new approach to automatically generate GUI test cases and validation points from a set of annotated use cases. This technique helps to reduce the effort required in GUI modeling and test coverage analysis during the software testing process. The test case generation process described in this paper is initially guided by use cases describing the GUI behavior, recorded as a set of interactions with the GUI elements (e.g., widgets being clicked, data input, etc.). These use cases (modeled as a set of initial test cases) are annotated by the tester to indicate interesting variations in widget values (ranges, valid or invalid values) and validation rules with expected results. Once the use cases are annotated, this approach uses the new defined values and validation rules to automatically generate new test cases and validation points, easily expanding the test coverage. Also, the process allows narrowing the GUI model testing to precisely identify the set of GUI elements, interactions, and values the tester is interested in.

Test path generation approach for GUI based on event weight

Focusing on the problem of infinite paths created by the events for GUI,the author presented a test path generation approach for GUI based on event weight in an event-flow graph(EFG).The event-flow graph model was analyzed,and Analytic Hierarchy Process(AHP) method was adopted to compute event weight.Test paths were generated by traversing event-flow graph on the basis of event weight.This approach simplifies the test paths to a certain extent,makes the test paths become more effective,and improves the GUI testing efficiency.

J2ME 기반 모바일 응용 소프트웨어 GUI 자동화 테스트 지원기

Recently, the most mobile application software developers are being faced by short developing time (fast time-to-market), various requirements and requirement changes. In order to overcome the environment, developer should support user friendly GUI and assure the quality from the early developing process using GUI test method. This paper proposes a test method and tool for mobile software GUI, which reduces test time and supports comfortable test environment by user oriented UI design. We implemented a tool based on test scenarios based image flow for high testing accuracy and test case reusability. And the results show the characteristics of the method compared to the existing tools.

Analysis models for the Underwater Sound Reference Division low-frequency acoustic calibration systems.

The state of an art for calibrating acoustic transducers at very low frequencies is by way of a confined and well understood environment. The Underwater Sound Reference Division (USRD) has three such calibration systems, called systems K, J, and L, respectively. Each system has a cylindrical tube, of certain length and diameter, that determine a cutoff calibration frequency. System K operates in a standing wave mode condition. Systems J and L both operate in traveling wave mode conditions, where plane waves propagate from one end of the tube to the other. Optimally locating a calibrated transducer and an unknown within the tube provides the proper configuration for calibration. This paper demonstrates the simulation tools to predict performance of aforementioned low‐frequency calibration systems. The mathematical model, the GUI coding, simulation, predicted, and test results will be presented.

이미지 플로우 기반의 모바일 GUI 테스트 도구에 관한 연구

모바일 어플리케이션 소프트웨어의 시장은 여러 소프트웨어 시장 중 가장 많은 어플리케이션 소프트웨어를 출시하고 있으며 모바일 어플리케이션에서 가장 중요한 사용자와의 정보 교환 수단으로는 GUI 가 있다. 현재까지의 모바일 어플리케이션에서의 GUI 테스트 방법으로는 테스터가 한단계, 한단계 버튼을 눌러가며 화면을 체크하는 원시적인 방법의 테스트가 이루어지고 있다. 이에 본 논문에서는 모바일 상에서 이루어지는 정적 화면 전환의 경우 테스트 수행 결과를 이미지 플로우 기반으로 표시함으로써 GUI를 테스트 하는 방법을 제시하고 테스트 커버리지까지 측정할 수 있는 방법을 제시한다. In order to enhance the productivity of mobile and develop reliable software, the test of mobile application software should be required absolutely. The most important way to communicate to users has been used to GUI. But GUI test method in mobile has no the test automation system except manual test by checklist. In this paper we present a test method and tool by image flow to reduce the time required and finds out errors to GUI, by carrying out the study on automatic GUI testing tool based on image flow to GUI test of mobile application.

Towards the Integration of Visual and Formal Models for GUI Testing

This paper presents an approach to diminish the effort required in GUI modelling and test coverage analysis within a model-based GUI testing process. A familiar visual notation a subset of UML with minor extensions is used to model the structure, behaviour and usage of GUIs at a high level of abstraction and to describe test adequacy criteria. The GUI visual model is translated automatically to a model-based formal specification language (e.g., Spec♯), hiding formal details from the testers. Then, additional behaviour may be added to the formal model to be used as a test oracle. The adequacy of the test cases generated automatically from the formal model is accessed based on the structural coverage of the UML behavioural diagrams.

Automated Oracle Based on Multi-Weighted Neural Networks for GUI Testing

Automated Oracle Based on Multi-Weighted Neural Networks for GUI Testing

An event‐flow model of GUI‐based applications for testing

AbstractGraphical user interfaces (GUIs) are by far the most popular means used to interact with today's software. The functional correctness of a GUI is required to ensure the safety, robustness and usability of an entire software system. GUI testing techniques used in practice are resource intensive; model‐based automated techniques are rarely employed. A key reason for the reluctance in the adoption of model‐based solutions proposed by researchers is their limited applicability; moreover, the models are expensive to create. Over the past few years, the present author has been developing different models for various aspects of GUI testing. This paper consolidates all of the models into one scalable event‐flow model and outlines algorithms to semi‐automatically reverse‐engineer the model from an implementation. Earlier work on model‐based test‐case generation, test‐oracle creation, coverage evaluation, and regression testing is recast in terms of this model by defining event‐space exploration strategies (ESESs) and creating an end‐to‐end GUI testing process. Three such ESESs are described: for checking the event‐flow model, test‐case generation, and test‐oracle creation. Two demonstrational scenarios show the application of the model and the three ESESs for experimentation and application in GUI testing. Copyright © 2007 John Wiley & Sons, Ltd.

A practical approach to testing GUI systems

GUI systems are becoming increasingly popular thanks to their ease of use when compared against traditional systems. However, GUI systems are often challenging to test due to their complexity and special features. Traditional testing methodologies are not designed to deal with the complexity of GUI systems; using these methodologies can result in increased time and expense. In our proposed strategy, a GUI system will be divided into two abstract tiers--the component tier and the system tier. On the component tier, a flow graph will be created for each GUI component. Each flow graph represents a set of relationships between the pre-conditions, event sequences and post-conditions for the corresponding component. On the system tier, the components are integrated to build up a viewpoint of the entire system. Tests on the system tier will interrogate the interactions between the components. This method for GUI testing is simple and practical; we will show the effectiveness of this approach by performing two empirical experiments and describing the results found.

Controlling Testing Using Three-Tier Model Architecture

In this paper, based on our earlier work, we introduce test model architecture for model-based testing through a GUI. The model architecture consists of three tiers. The tiers separate important concerns in GUI testing: navigation in the GUI using keywords, high-level concepts expressed as action words, and test-control related issues defined as control words. For test control, we define a novel coverage language to express coverage objectives. Furthermore, we introduce our refined vision for the associated tool platform enabling system level testing in the Symbian environment. The architecture includes a commercial GUI testing tool that we have extended with components enabling the use of test models.