Graphical User Interface Testing Research Articles

Constrained Interaction Testing: A Systematic Literature Study

Interaction testing can be used to effectively detect faults that are otherwise difficult to find by other testing techniques. However, in practice, the input configurations of software systems are subjected to constraints, especially in the case of highly configurable systems. Handling constraints effectively and efficiently in combinatorial interaction testing is a challenging problem. Nevertheless, researchers have attacked this challenge through different techniques, and much progress has been achieved in the past decade. Thus, it is useful to reflect on the current achievements and shortcomings and to identify potential areas of improvements. This paper presents the first comprehensive and systematic literature study to structure and categorize the research contributions for constrained interaction testing. Following the guidelines of conducting a literature study, the relevant data are extracted from a set of 103 research papers belonging to constrained interaction testing. The topics addressed in constrained interaction testing research are classified into four categories of constraint test generation, application, generation and application, and model validation studies. The papers within each of these categories are extensively reviewed. Apart from answering several other research questions, this paper also discusses the applications of constrained interaction testing in several domains, such as software product lines, fault detection and characterization, test selection, security, and graphical user interface testing. This paper ends with a discussion of limitations, challenges, and future work in the area.

The experimental applications of search-based techniques for model-based testing: Taxonomy and systematic literature review

ContextModel-based testing (MBT) aims to generate executable test cases from behavioral models of software systems. MBT gains interest in industry and academia due to its provision of systematic, automated, and comprehensive testing. Researchers have successfully applied search-based techniques (SBTs) by automating the search for an optimal set of test cases at reasonable cost compared to other more expensive techniques. Thus, there is a recent surge toward the applications of SBTs for MBT because the generated test cases are optimal and have low computational cost. However, successful, future SBTs for MBT applications demand deep insight into its existing experimental applications that underlines stringent issues and challenges, which is lacking in the literature. ObjectiveThe objective of this study is to comprehensively analyze the current state-of-the-art of the experimental applications of SBTs for MBT and present the limitations of the current literature to direct future research. MethodWe conducted a systematic literature review (SLR) using 72 experimental papers from six data sources. We proposed a taxonomy based on the literature to categorize the characteristics of the current applications. ResultsThe results indicate that the majority of the existing applications of SBTs for MBT focus on functional and structural coverage purposes, as opposed to stress testing, regression testing and graphical user interface (GUI) testing. We found research gaps in the existing applications in five areas: applying multi-objective SBTs, proposing hybrid techniques, handling complex constraints, addressing data and requirement-based adequacy criteria, and adapting landscape visualization. Only twelve studies proposed and empirically evaluated the SBTs for complex systems in MBT. ConclusionThis extensive systematic analysis of the existing literature based on the proposed taxonomy enables to assist researchers in exploring the existing research efforts and reveal the limitations that need additional investigation.

효과적인 모델 기반 안드로이드 GUI 테스팅을 위한 GUI 상태 비교 기법

Graphical user interface testing (GUI testing) techniques have been widely used to test the functionality of Android applications (apps) and to detect faults for verification of the reliability and usability of apps. To adequately test the behaviors of apps, a number of studies on model-based GUI testing techniques have been performed on Android apps. However, the effectiveness of model-based techniques greatly depends on the quality of the GUI model, because model-based GUI testing techniques generate test inputs based on this model. Therefore, in order to improve testing effectiveness in model-based techniques, accurate and efficient GUI model generation has to be achieved using an improved model generation technique with concrete definition of GUI states. For accurate and efficient generation of a GUI model and test inputs, this study suggests a hierarchical GUI state comparison technique and evaluates this technique through comparison with the existing model-based techniques, considering activities as GUI states. Our results show that the proposed technique outperforms existing approaches and has the potential to improve the performance of model-based GUI testing techniques for Android apps.

MPBEC, a Matlab Program for Biomolecular Electrostatic Calculations

One of the most used and efficient approaches to compute electrostatic properties of biological systems is to numerically solve the Poisson–Boltzmann (PB) equation. There are several software packages available that solve the PB equation for molecules in aqueous electrolyte solutions. Most of these software packages are useful for scientists with specialized training and expertise in computational biophysics. However, the user is usually required to manually take several important choices, depending on the complexity of the biological system, to successfully obtain the numerical solution of the PB equation. This may become an obstacle for researchers, experimentalists, even students with no special training in computational methodologies. Aiming to overcome this limitation, in this article we present MPBEC, a free, cross-platform, open-source software that provides non-experts in the field an easy and efficient way to perform biomolecular electrostatic calculations on single processor computers. MPBEC is a Matlab script based on the Adaptative Poisson–Boltzmann Solver, one of the most popular approaches used to solve the PB equation. MPBEC does not require any user programming, text editing or extensive statistical skills, and comes with detailed user-guide documentation. As a unique feature, MPBEC includes a useful graphical user interface (GUI) application which helps and guides users to configure and setup the optimal parameters and approximations to successfully perform the required biomolecular electrostatic calculations. The GUI also incorporates visualization tools to facilitate users pre- and post-analysis of structural and electrical properties of biomolecules. Program summaryProgram title: MPBECCatalogue identifier: AEXW_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEXW_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: UTSA licenseNo. of lines in distributed program, including test data, etc.: 1505214No. of bytes in distributed program, including test data, etc.: 115585507Distribution format: tar.gzProgramming language: Matlab R2010a and higher versions.Computer: Any computer at least with 2.5 GHz speed.Operating system: Windows, Linux Fedora, and Mac OSX.RAM: At least 2 GB of free memory.Classification: 3, 10.External routines: Java, Jmol, pdb2pqr, Propka.Nature of problem: Numerical solution of the linearized PB equation.Solution method: MPBEC uses sparse matrix calculations and iterative linear solvers (gmres, minres, and bicgstab) to solve the discretized (box method) linear PB equation. The stability and the convergence of these iterative solvers are improved by using the incomplete LU factorization which generates pre-conditioner matrices L (lower triangular matrix) and U (upper triangular matrix) at low computational cost.Unusual features: MPBEC incorporates a GUI to provide non-experts in computational biophysics a user-friendly and intuitive tool to obtain biomolecular electrostatic calculations. The GUI provides helpful information about how to fill out the input data by moving the mouse pointer over the corresponding text or blank box. The GUI tests all the input data before running MPBEC to make sure that the solver is properly configured and facilitates the user with the pre- and post-analysis of the biomolecular calculations.Additional comments: MPBEC checks available and required RAM memory before allocation to alert users when the available RAM memory is insufficient to perform the required calculations depending basically on the number of grid points and the size and number of atoms of the biomolecule(s). Therefore, MPBEC might be used with lower RAM memory and processor speed conditions than those mentioned above. The distribution file for this program is over 115 Mbytes and therefore is not delivered directly when download or Email is requested. Instead a html file giving details of how the program can be obtained is sent.Running time: The software runs on single processor computers at low-to-moderate computational cost depending on the computer performance, the biomolecule size, the grid resolution, and required calculations.

Visual GUI testing in practice

In today's software development industry, high-level tests such as Graphical User Interface (GUI) based system and acceptance tests are mostly performed with manual practices that are often costly,...

An Observed Model Approach for Testing GUI Model Driven Interface in Water-Flow Model

Graphical-user interface (GUI) -based software applications are with the job of verifying that these tasks can be performed using the software; and that the software does not “behave badly”. A set of use cases with high-level descriptions is also given to the testers. The tester executes these high-level steps by using GUI widgets on which events can be performed. GUI testers are both automated and manual working with undetermined input spaces. The testers unknowingly miss the event sequences navigated by the GUI, and fail to realize its implementation which may allow the execution of some disallowed sequences. This thesis proposes Preemptive Regression Testing (PRT) an adaptive taxing move to address this challenge. Whenever a change in the exposure of any service artifact is detected, PRT recursively preempts the current setting of regression test and creates a sub-session of the current test session to assure lately identified changes in coverage by adjusting the precedence of the test cases in the test suite. Then, the execution will resume the sub session from the balanced location of the test cases. PRT terminates only when each test case in the test suite has been executed at least once without any preemption activated in between any test case executions. The new result confirms that testing water flow method based web service in the phase of such changes is very difficult and one of the PRT-enriched techniques will overcome the challenge.

Model-Based Contract Testing of Graphical User Interfaces

Graphical User Interfaces (GUIs) are critical for the security, safety and reliability of software systems. Injection attacks, for instance via SQL, succeed due to insufficient input validation and can be avoided if contract-based approaches, such as Design by Contract, are followed in the software development lifecycle of GUIs. This paper proposes a model-based testing approach for detecting GUI data contract violations, which may result in serious failures such as system crash. A contract-based model of GUI data specifications is used to develop test scenarios and to serve as test oracle. The technique introduced uses multi terminal binary decision diagrams, which are designed as an integral part of decision table-augmented event sequence graphs, to implement a GUI testing process. A case study, which validates the presented approach on a port scanner written in Java programming language, is presented.

Event-Coverage and Weight based Method for Test Suite Prioritization

There are many challenges in testing of Graphical User Interface (GUI) applications due to its event driven nature and infinite input domain. Testing each and every possible combination of input require creating number of test cases to satisfy the adequacy criteria of GUI testing. It is not possible to test each and every test case within specified time frame. Therefore it is important to assign higher priority to test cases which have higher fault revealing capability than other test cases. Various methods are specified in literature for test suite prioritization of GUI based software and some of them are based on interaction coverage and weight of events. Weight based methods are defined namely fault prone weight based method, random weight based method and equal weight based method in which fault prone based method is most effective. In this paper we have proposed Event-Coverage and Weight based Method (EC-WBM) which prioritizes GUI test cases according to their event coverage and weight value. Weight value will be assigned based on unique event coverage and fault revealing capability of events. Event coverage based method is used to evaluate the adequacy of test cases. EC-WBM is evaluated for 2 applications one is Notepad and another is Calculator. Fault seeding method is used to create number of versions of application and these faults are evaluated using APFD (Average percentage of fault detection). APFD for prioritized test cases of Notepad is 98% and APFD for non-prioritized test cases is 62%.

Research on the GUI Testing Based on the Bayesian Network Model

Graphical User Interfaces (GUI) is becoming increasingly important in the software field, while GUI testing is becoming the key issues restricting GUI rapid development. GUI testing mainly serves two goals: to establish confidence in assessment of GUI and find more software defects in GUI testing. For this purpose, any testing method must be better at detecting defects. However, GUI testing is faced with many challenges due to the immense number of event interactions. In this paper, it introduces a Bayesian model guiding the process of GUI testing, discusses the Bayesian model topology and its issues encountered in the modeling process. In the end, a case reveals the validity of the GUI testing model.

Generating combinatorial test cases using Simplified Swarm Optimization (SSO) algorithm for automated GUI functional testing

Graphical User Interface (GUI) is the outer skin of programs that facilitate the interaction between the user and different type of computing devices. It is been used in different aspects ranging from normal computers, mobile device, to even very small device nowadays like watches. This interaction uses different tools and programming objects like images, text, buttons, checkboxes, etc. With this emergence of different types of GUIs, they become an essential component to be tested (if available in the software) to ensure that the software meets the required quality by the user. In contrast to non-functional testing, function testing of GUI insures a proper interaction between the user and the application interface without dealing with the coding internals. In this paper, a strategy for GUI functional testing using Simplified Swarm Optimization (SSO) is proposed. The SSO is used to generate an optimized test suite with the help of Event-Interaction Graph (EIG). The proposed strategy also manages and repairs the test suites by deleting the unnecessary event sequences that are not applicable. The proposed generation algorithm based on SSO has proved its effectiveness by evaluating it against other algorithms. In addition, the strategy is applied on a standard case study and proved its applicability in reality.

Comparative Study of Open Source Capture and Replay Tools

Graphical user interface (GUI) is becoming more and more important due to its widespread use in software products and computers. However, testing GUI still remains a big challenge to ensure the quality of the system and therefore to improve the user satisfaction. Manual testing permits ad hoc testing, nevertheless it often results in large amounts of manual labor and thus increases cost. Using capture and replay tools to support the testing process helps to overcome the problems of the manual testing. In this paper we compare five open source capture and replay tools in terms of ease of use, and capture and replay features. We compare the tools Abbot, Jacareto, JFCUnit, Marathon and Pounder, assessing generic, and capture and replay characteristics. After evaluating each tool, we selected the one that showed the best results in almost all criteria. The findings of this study may serve as guidance for novice testers or companies that consider employing open source capture and replay tools to automate the GUI testing process.

An Observe-Model-Exercise* Paradigm to Test Event-Driven Systems with Undetermined Input Spaces

System testing of software applications with a graphical-user interface (GUI) front-end requires that sequences of GUI events, that sample the application’s input space, be generated and executed as test cases on the GUI. However, the context-sensitive behavior of the GUI of most of today’s non-trivial software applications makes it practically impossible to fully determine the software’s input space. Consequently, GUI testers—both automated and manual—working with undetermined input spaces are, in some sense, blindly navigating the GUI, unknowingly missing allowable event sequences, and failing to realize that the GUI implementation may allow the execution of some disallowed sequences. In this paper, we develop a new paradigm for GUI testing, one that we call Observe-Model-Exercise* (OME*) to tackle the challenges of testing context-sensitive GUIs with undetermined input spaces. Starting with an incomplete model of the GUI’s input space, a set of coverage elements to test, and test cases, OME* iteratively observes the existence of new events during execution of the test cases, expands the model of the GUI’s input space, computes new coverage elements, and obtains new test cases to exercise the new elements. Our experiment with 8 open-source software subjects, more than 500,000 test cases running for almost 1,100 machine-days, shows that OME* is able to expand the test space on average by 464.11 percent; it detected 34 faults that had never been detected before.

Visual GUI testing in practice: challenges, problemsand limitations

In today's software development industry, high-level tests such as Graphical User Interface (GUI) based system and acceptance tests are mostly performed with manual practices that are often costly, tedious and error prone. Test automation has been proposed to solve these problems but most automation techniques approach testing from a lower level of system abstraction. Their suitability for high-level tests has therefore been questioned. High-level test automation techniques such as Record and Replay exist, but studies suggest that these techniques suffer from limitations, e.g. sensitivity to GUI layout or code changes, system implementation dependencies, etc. Visual GUI Testing (VGT) is an emerging technique in industrial practice with perceived higher flexibility and robustness to certain GUI changes than previous high-level (GUI) test automation techniques. The core of VGT is image recognition which is applied to analyze and interact with the bitmap layer of a system's front end. By coupling image recognition with test scripts, VGT tools can emulate end user behavior on almost any GUI-based system, regardless of implementation language, operating system or platform. However, VGT is not without its own challenges, problems and limitations (CPLs) but, like for many other automated test techniques, there is a lack of empirically-based knowledge of these CPLs and how they impact industrial applicability. Crucially, there is also a lack of information on the cost of applying this type of test automation in industry. This manuscript reports an empirical, multi-unit case study performed at two Swedish companies that develop safety-critical software. It studies their transition from manual system test cases into tests automated with VGT. In total, four different test suites that together include more than 300 high-level system test cases were automated for two multi-million lines of code systems. The results show that the transitioned test cases could find defects in the tested systems and that all applicable test cases could be automated. However, during these transition projects a number of hurdles had to be addressed; a total of 58 different CPLs were identified and then categorized into 26 types. We present these CPL types and an analysis of the implications for the transition to and use of VGT in industrial software development practice. In addition, four high-level solutions are presented that were identified during the study, which would address about half of the identified CPLs. Furthermore, collected metrics on cost and return on investment of the VGT transition are reported together with information about the VGT suites' defect finding ability. Nine of the identified defects are reported, 5 of which were unknown to testers with extensive experience from using the manual test suites. The main conclusion from this study is that even though there are many challenges related to the transition and usage of VGT, the technique is still valuable, flexible and considered cost-effective by the industrial practitioners. The presented CPLs also provide decision support in the use and advancement of VGT and potentially other automated testing techniques similar to VGT, e.g. Record and Replay.

A Unified Bug Testing Software for Biological and Other Systems

A Unified Bug Testing Software for Biological and Other Systems Automatic bug testing is very crucial for software systems that are widely used. This is especially true when the system is very complex. Unexpected and undesired events could happen due to the presence of bugs. For instance, a web system might take a very long time on a specific query because of the presence of a bug. We cannot expect the creators of the web system to be continually monitoring the system looking for such events. It will greatly benefit the creators of the web system if such events can be automatically detected and if they are notified in a timely fashion. In this paper we report one such bug testing software that we have created for the MnM web system for motif search. MnM is widely used by biologists all over the globe. We feel that the architecture we have employed for our bug testing software is generic and can be applied to other software systems as well. There exist numerous conventional bug testing software tools which are designed to perform only functional or graphical user interface testing.

Guided GUI testing of android apps with minimal restart and approximate learning

Smartphones and tablets with rich graphical user interfaces (GUI) are becoming increasingly popular. Hundreds of thousands of specialized applications, called apps, are available for such mobile platforms. Manual testing is the most popular technique for testing graphical user interfaces of such apps. Manual testing is often tedious and error-prone. In this paper, we propose an automated technique, called Swift-Hand , for generating sequences of test inputs for Android apps. The technique uses machine learning to learn a model of the app during testing, uses the learned model to generate user inputs that visit unexplored states of the app, and uses the execution of the app on the generated inputs to refine the model. A key feature of the testing algorithm is that it avoids restarting the app, which is a significantly more expensive operation than executing the app on a sequence of inputs. An important insight behind our testing algorithm is that we do not need to learn a precise model of an app, which is often computationally intensive, if our goal is to simply guide test execution into unexplored parts of the state space. We have implemented our testing algorithm in a publicly available tool for Android apps written in Java. Our experimental results show that we can achieve significantly better coverage than traditional random testing and L*-based testing in a given time budget. Our algorithm also reaches peak coverage faster than both random and L*-based testing.

GUITAR: an innovative tool for automated testing of GUI-driven software

Most of today's software applications feature a graphical user interface (GUI) front-end. System testing of these applications requires that test cases, modeled as sequences of GUI events, be generated and executed on the software. We term GUI testing as the process of testing a software application through its GUI. Researchers and practitioners agree that one must employ a variety of techniques (e.g., model-based, capture/replay, manually scripted) for effective GUI testing. Yet, the tools available today for GUI testing are limited in the techniques they support. In this paper, we describe an innovative tool called GUITAR that supports a wide variety of GUI testing techniques. The innovation lies in the architecture of GUITAR, which uses plug-ins to support flexibility and extensibility. Software developers and quality assurance engineers may use this architecture to create new toolchains, new workflows based on the toolchains, and plug in a variety of measurement tools to conduct GUI testing. We demonstrate these features of GUITAR via several carefully crafted case studies.

Study Paper on Test Case Generation for Gui Based Testing

With the advent of WWW and outburst in technology and software development, testing the software became a major concern. Due to the importance of the testing phase in a software development life cycle, testing has been divided into graphical user interface (GUI) based testing, logical testing, integration testing, etc.GUI Testing has become very important as it provides more sophisticated way to interact with the software. The complexity of testing GUI increased over time. The testing needs to be performed in a way that it provides effectiveness, efficiency, increased fault detection rate and good path coverage. To cover all use cases and to provide testing for all possible (success/failure) scenarios the length of the test sequence is considered important. Intent of this paper is to study some techniques used for test case generation and process for various GUI based software applications.

Test Case PrioritizationUsing Fuzzy Logic for GUI based Software

Testing of GUI (Graphical User Interface) applications has many challenges due to its event driven nature and infinite input domain. It is very difficult for any programmer to test for each and every possible input. When test cases are generated using automated testing tool it uses each and every possible combination to generate test cases hence generates numerous number of test case for any GUI based application. Within a defined time frame it is not possible to test every test case, that is why test cases prioritization is required. Test-case prioritization has been widely proposed and used in recent years as it can improve the rate of fault detection during the testing phase. Very few methods are defined for GUI Test case prioritization that usually consider single criteria for assigning priority for the test case which is not sufficient for the consideration of that test case as more fault revealing. In this paper we have proposed a method for assigning weight value on the basis of multiple factors as one of the criteria for test case prioritization for GUI based software. These factors are: The type of event, Event Interaction, and Parameter-value interaction coverage-based criteria. In the proposed approach priority is assigned based upon these factors using fuzzy logic model. Experimental results indicate that the proposed model is suitable for prioritizing the test cases of GUI based software.

Apply ant colony to event-flow model for graphical user interface test case generation

Aimed at the complex and inefficient testing issues of graphical user interface (GUI) software, this study describes an automatic framework for GUI test case generation. The framework includes a reverse engineering of executive GUI to create GUI information and event-flow model. According to these information and model, an ant colony algorithm is applied to generate test cases. The study shows that by identifying valid event interactions, the generated test cases are capable of finding deep faults without infeasible problems; the configuration of different parameters for ant colony and the good design of the proposed framework provide flexible interfaces to control test suites.

Analysis of OCR Design and Implementation for GUI Modeling

Many efforts have been taken to achieve automated Graphical User Interface (GUI) testing. The most popular way is model-based testing, which supports automated test case generation and execution. But building such a model is a non-trivial task, which usually costs the most work-load in the entire testing process. Most of the approaches about automated model deriving are dependant on the programming language or specific OS. In this paper, we proposed a new approach of GUI modeling using Optical Character Recognition (OCR), and technical poblems encountered have been analyzed in deatail. Case study shows that our approach is capable of analyzing most of the GUI windows, and generating corresponding model and hence eliminates the above constraint.