Interface Testing Research Articles

Constrained Interaction Testing: A Systematic Literature Study

Interaction testing can be used to effectively detect faults that are otherwise difficult to find by other testing techniques. However, in practice, the input configurations of software systems are subjected to constraints, especially in the case of highly configurable systems. Handling constraints effectively and efficiently in combinatorial interaction testing is a challenging problem. Nevertheless, researchers have attacked this challenge through different techniques, and much progress has been achieved in the past decade. Thus, it is useful to reflect on the current achievements and shortcomings and to identify potential areas of improvements. This paper presents the first comprehensive and systematic literature study to structure and categorize the research contributions for constrained interaction testing. Following the guidelines of conducting a literature study, the relevant data are extracted from a set of 103 research papers belonging to constrained interaction testing. The topics addressed in constrained interaction testing research are classified into four categories of constraint test generation, application, generation and application, and model validation studies. The papers within each of these categories are extensively reviewed. Apart from answering several other research questions, this paper also discusses the applications of constrained interaction testing in several domains, such as software product lines, fault detection and characterization, test selection, security, and graphical user interface testing. This paper ends with a discussion of limitations, challenges, and future work in the area.

Прикладное применение динамического анализа программ, исполняющихся в интерпретирующих средах

The present-day trends in software engineering include the steady increase of code and design complexity which reinforces the high demand in automated software testing and analysis tools. In this paper, we showcase several dynamic program analysis applications and present our solutions. These applications include memory profiling, automated test generation using dynamic symbolic execution and automated detection of concurrency bugs in multithreaded programs. Our memory profiling tool is designed for Java applications for Android and it is implemented through Android Dalvik VM modification. This approach allowed us to overcome existing Dalvik VM limitations that make existing profiling tools based on dynamic bytecode instrumentation inaccessible. We have successfully applied our tool to several core Android applications - the results provided in the paper outline the effectiveness of the approach. The second solution we discuss in the paper - dynamic symbolic execution for test generation automation - allows us to efficiently generate test scenarios for Java program graphical user interface. The core technologies of the approach include the use of static bytecode instrumentation and automatic GUI model extraction. We implement the approach on top of a user interface test automation framework GUITAR. Finally, we present our approach to automatically identify concurrency bugs in multithreaded Java applications. The approach is based on static bytecode instrumentation for trace generation and employs ThreadSanitizer defect detection tool for identifying bugs.

Automatic GUI test by using SIFT matching

In software development process, the last step is usually the Graphic User Interface(GUI) test, which is part of the final user experience (UE) test. Traditionally, there exist some GUI test tools in the market, such as Abbot Java GUI Test Framework and Pounder, in which testers pre-configure in the script all desired actions and instructions for the computer, nonetheless requiring too much of invariance of GUI environment; and they require reconfiguration in case of GUI changes, therefore still to be done mostly manually and hard for non-programmer testers to. Consequently, we proposed GUI tests by image recognition to automate the last process; we managed to innovate upon current algorithms such as SIFT and Random Fern, from which we develop the new algorithm scheme retrieving most efficient feature and dispelling inefficient part of each algorithm. Computers then apply the algorithm, to search for target patterns themselves and take subsequent actions such as manual mouse, keyboard and screen I/O automatically to test the GUI without any manual instructions. Test results showed that the proposed approach can accelerate GUI test largely compared to current benchmarks.

The experimental applications of search-based techniques for model-based testing: Taxonomy and systematic literature review

ContextModel-based testing (MBT) aims to generate executable test cases from behavioral models of software systems. MBT gains interest in industry and academia due to its provision of systematic, automated, and comprehensive testing. Researchers have successfully applied search-based techniques (SBTs) by automating the search for an optimal set of test cases at reasonable cost compared to other more expensive techniques. Thus, there is a recent surge toward the applications of SBTs for MBT because the generated test cases are optimal and have low computational cost. However, successful, future SBTs for MBT applications demand deep insight into its existing experimental applications that underlines stringent issues and challenges, which is lacking in the literature. ObjectiveThe objective of this study is to comprehensively analyze the current state-of-the-art of the experimental applications of SBTs for MBT and present the limitations of the current literature to direct future research. MethodWe conducted a systematic literature review (SLR) using 72 experimental papers from six data sources. We proposed a taxonomy based on the literature to categorize the characteristics of the current applications. ResultsThe results indicate that the majority of the existing applications of SBTs for MBT focus on functional and structural coverage purposes, as opposed to stress testing, regression testing and graphical user interface (GUI) testing. We found research gaps in the existing applications in five areas: applying multi-objective SBTs, proposing hybrid techniques, handling complex constraints, addressing data and requirement-based adequacy criteria, and adapting landscape visualization. Only twelve studies proposed and empirically evaluated the SBTs for complex systems in MBT. ConclusionThis extensive systematic analysis of the existing literature based on the proposed taxonomy enables to assist researchers in exploring the existing research efforts and reveal the limitations that need additional investigation.

효과적인 모델 기반 안드로이드 GUI 테스팅을 위한 GUI 상태 비교 기법

Graphical user interface testing (GUI testing) techniques have been widely used to test the functionality of Android applications (apps) and to detect faults for verification of the reliability and usability of apps. To adequately test the behaviors of apps, a number of studies on model-based GUI testing techniques have been performed on Android apps. However, the effectiveness of model-based techniques greatly depends on the quality of the GUI model, because model-based GUI testing techniques generate test inputs based on this model. Therefore, in order to improve testing effectiveness in model-based techniques, accurate and efficient GUI model generation has to be achieved using an improved model generation technique with concrete definition of GUI states. For accurate and efficient generation of a GUI model and test inputs, this study suggests a hierarchical GUI state comparison technique and evaluates this technique through comparison with the existing model-based techniques, considering activities as GUI states. Our results show that the proposed technique outperforms existing approaches and has the potential to improve the performance of model-based GUI testing techniques for Android apps.

메모리 반도체 검사 장비 인터페이스를 위한 크로스플랫폼 소프트웨어 기술

메모리 반도체 패키지 검사 공정에서 TIS(Tester Interface Software)는 디바이스가 검사 장비에 투입 될 때부터 배출될 때까지 검사 장비가 디바이스 검사를 진행하는데 필요한 모든 소프트웨어 기능을 제공한다. 하지만, 공정에서 사용되는 장비와 장비를 제어하기 위한 컴퓨터 및 운영체제의 종류가 다양하여 동일한 기능을 수행해야 하는 TIS가 테스터 장비마다 독립적으로 개발 및 운영되고 있다. 이는 많은 시간과 비용을 요구할 뿐만 아니라 소프트웨어의 품질에도 많은 영향을 미치고 있으며, 이러한 문제는 추가되는 장비의 종류가 증가할수록 심화될 것이다. 본 논문에서는 이러한 문제를 해결하기 위해 이종 장비와 운영체제에 적용 가능한 CTIS(Cross-platform Tester Interface Software)을 제안한다.

Mobile Diagnosis System with Emergency Telecare in Thailand (MOD-SET)

Abstract This paper presents a Mobile Diagnosis System with Emergency Telecare functionality applied in Community Health Centers and hospitals in Thailand. The system results from an integration of two previously released applications: a tele-diagnosis system (TEDIST) and a smartphone based emergency report system (SPEARS). The functionality of these separate systems have been integrated and enhanced with a speech-to-text function, which users (or potential patients) apply to communicate with the system by speaking into their phones. A rule-based expert system is controlled by medical professionals regarding the automated diagnosis results. The emergency response component can communicate with local emergency services from hospitals and other organizations in order to find the nearest facility for help and support in case of emergency. Tests of the user interface and system tests have been carried out and have shown encouraging results.

Event-Coverage and Weight based Method for Test Suite Prioritization

There are many challenges in testing of Graphical User Interface (GUI) applications due to its event driven nature and infinite input domain. Testing each and every possible combination of input require creating number of test cases to satisfy the adequacy criteria of GUI testing. It is not possible to test each and every test case within specified time frame. Therefore it is important to assign higher priority to test cases which have higher fault revealing capability than other test cases. Various methods are specified in literature for test suite prioritization of GUI based software and some of them are based on interaction coverage and weight of events. Weight based methods are defined namely fault prone weight based method, random weight based method and equal weight based method in which fault prone based method is most effective. In this paper we have proposed Event-Coverage and Weight based Method (EC-WBM) which prioritizes GUI test cases according to their event coverage and weight value. Weight value will be assigned based on unique event coverage and fault revealing capability of events. Event coverage based method is used to evaluate the adequacy of test cases. EC-WBM is evaluated for 2 applications one is Notepad and another is Calculator. Fault seeding method is used to create number of versions of application and these faults are evaluated using APFD (Average percentage of fault detection). APFD for prioritized test cases of Notepad is 98% and APFD for non-prioritized test cases is 62%.

A Unified Bug Testing Software for Biological and Other Systems

A Unified Bug Testing Software for Biological and Other Systems Automatic bug testing is very crucial for software systems that are widely used. This is especially true when the system is very complex. Unexpected and undesired events could happen due to the presence of bugs. For instance, a web system might take a very long time on a specific query because of the presence of a bug. We cannot expect the creators of the web system to be continually monitoring the system looking for such events. It will greatly benefit the creators of the web system if such events can be automatically detected and if they are notified in a timely fashion. In this paper we report one such bug testing software that we have created for the MnM web system for motif search. MnM is widely used by biologists all over the globe. We feel that the architecture we have employed for our bug testing software is generic and can be applied to other software systems as well. There exist numerous conventional bug testing software tools which are designed to perform only functional or graphical user interface testing.

Apply ant colony to event-flow model for graphical user interface test case generation

Aimed at the complex and inefficient testing issues of graphical user interface (GUI) software, this study describes an automatic framework for GUI test case generation. The framework includes a reverse engineering of executive GUI to create GUI information and event-flow model. According to these information and model, an ant colony algorithm is applied to generate test cases. The study shows that by identifying valid event interactions, the generated test cases are capable of finding deep faults without infeasible problems; the configuration of different parameters for ant colony and the good design of the proposed framework provide flexible interfaces to control test suites.

Analysis of OCR Design and Implementation for GUI Modeling

Many efforts have been taken to achieve automated Graphical User Interface (GUI) testing. The most popular way is model-based testing, which supports automated test case generation and execution. But building such a model is a non-trivial task, which usually costs the most work-load in the entire testing process. Most of the approaches about automated model deriving are dependant on the programming language or specific OS. In this paper, we proposed a new approach of GUI modeling using Optical Character Recognition (OCR), and technical poblems encountered have been analyzed in deatail. Case study shows that our approach is capable of analyzing most of the GUI windows, and generating corresponding model and hence eliminates the above constraint.

Using ontology to generate test cases for GUI testing

Graphical User Interface (GUI) testing is a knowledge-intensive process. In this paper, ontology is introduced to generate user-centric GUI test cases. First, GUI and non-GUI components are captured by reverse engineering techniques. Next, relations among GUI components are analysed and a GUI ontology is constructed by representing all analysed results. Then, test case generation rules are defined and used to simplify test cases. After that, a case study is demonstrated on a general communication application, which shows that the proposed approach is technically feasible and ontology can facilitate GUI testing by utilising knowledge of GUI systems and experience of testers.

From formal requirements to automated web testing and prototyping

Use cases and user interface prototypes are agile techniques used to specify required functionalities of an object-oriented system. Testing can find most errors of the software and ensure that the requirement specifications are satisfied by the application. We can derive test cases from use cases and run them in user interfaces, but we do it in a mechanical way. In this paper, we propose a new approach for automating the generation of user interface prototypes and test cases for web applications. Starting from a formalization of the requirements based on controlled use cases extended with the system glossary and the user interface specifications, we automatically generate web pages and test cases which are passed as parameters for an automated web testing tool.

Agent based User Interface Test Automation for Silverlight Applications

Agent based User Interface Test Automation for Silverlight Applications

TEST AUTOMATION FOR NEW GENERATION XAML WEB APPLICATIONS USING MULTI AGENT SYSTEM

New generation of Interactive and attractive dynamic web applications are developed by Silverlight Technology. It is developed by Microsoft on Dot Net 3.5 Frame work using eXtended Application Markup Language (XAML). User Interface Test Automation for Silverlight Applications plays a vital role in software industry; especially User Interface Test Automation (UITA) in new technology like Silverlight is a challenging task because of its high security and low accessibility. Agent Technology is intermediate software that provides a better bridge between User Interface Test Automation and Silverlight applications. There are many different software agents for each group of controls to do User Interface Test Automation. For example button control agent will take care of all buttons in User Interface Test Automation. In this paper we present the XAML User Interface test automation using Multi Agent System. The main goal of Test Automation for New Generation XAML Web Applications Using Multi Agent System are to reduce cost and time to perform User Interface Test Automation.

A Graphical User Interface (GUI) Testing Methodology

Software testing in general and graphical user interface (GUI) testing in particular is one of the major challenges in the lifecycle of any software system. GUI testing is inherently more difficult than the traditional and command-line interface testing. Some of the factors that make GUI testing different from the traditional software testing and significantly more difficult are: a large number of objects, different look and feel of objects, many parameters associated with each object, progressive disclosure, complex inputs from multiple sources, and graphical outputs. The existing testing techniques for the creation and management of test suites need to be adapted/enhanced for GUIs, and new testing techniques are desired to make the creation and management of test suites more efficient and effective. In this article, a methodology is proposed to create test suites for a GUI. The proposed methodology organizes the testing activity into various levels. The tests created at a particular level can be reused at higher levels. This methodology extends the notion of modularity and reusability to the testing phase. The organization and management of the created test suites resembles closely to the structure of the GUI under test.

Model-based User Interface Testing With Spec Explorer and ConcurTaskTrees

Analytic usability analysis methods have been proposed as an alternative to user testing in early phases of development due to the cost of the latter approach. By working with models of the systems, analytic models are not capable of identifying implementation related problems that might have an impact on usability. Model-based testing enables the testing of an implemented software artefact against a model of what it should be (the oracle). In the case of model-based user interface testing, the models should be expressed at an adequate level of abstraction, adequately modelling the interaction process. This paper describes an effort to develop tool support enabling the use of task models as oracles for model-based testing of user interfaces.

Development of an Improved GUI Automation Test System Based on Event-Flow Graph

A more automated graphic user interface (GUI) test model, which is based on the event-flow graph, is proposed. In the model, a user interface automation API tool is first used to carry out reverse engineering for a GUI test sample so as to obtain the event-flow graph. Then two approaches are adopted to create GUI test sample cases. That is to say, an improved ant colony optimization (ACO) algorithm is employed to establish a sequence of testing cases in the course of the daily smoke test. The sequence goes through all object event points in the event-flow graph. On the other hand, the spanning tree obtained by deep breadth-first search (BFS) approach is utilized to obtain the testing cases from goal point to outset point in the course of the deep regression test. Finally, these cases are applied to test the new GUI. Moreover, according to the above-mentioned model, a corresponding prototype system based on Microsoft UI automation framework is developed, thus giving a more effective way to improve the GUI automation test in Windows OS.

ICANDO: Low cost multimodal interface for hand disabled people

The article presents the multimodal user interface ICANDO (Intellectual Computer AssistaNt for Disabled Operators) that was awarded with the first prize of the Loco Mummy Software Contest in 2006. The interface is intended mainly for assistance to the persons without hands or with disabilities of their hands or arms but could be useful for ordinary users at hands-free contactless human-computer interaction too. It combines the module for automatic recognition of voice commands in English, French and Russian as well as the head tracking module in one multimodal interface. ICANDO interface was applied for hands-free work with Graphical User Interface of a personal computer in such tasks as Internet communication and work with graphical and text documents. The article describes the aim and the architecture of the interface, the methods for speech recognition and head tracking, information fusion and synchronization of the multimodal streams. The presented results of testing and exploitation of ICANDO user interface have confirmed high accuracy and robustness of the interface for contactless operation with a computer. The comparison of multimodal and standard ways of interaction has discovered that the first one is slower by a factor of 1.9 that is quite well for hands-free interaction between a computer and an impaired person.

웹 어플리케이션의 사용자 인터페이스 테스트 자동화 기법

인터넷의 급속한 성장과 웹 관련 기술이 더 복잡해지면서 웹 응용 소프트웨어의 품질과 신뢰성이 중요하게 되었다. 웹 기반 소프트웨어는 전 세계에 흩어져 있는 수많은 사용자들을 대상으로 한다는 면에서 품질과 시험 방법 또한 중요하다. 이 논문은 브라우저 객체를 이용하여 HTML 웹 페이지의 사용자 입력 부분에 대한 테스트 케이스를 자동적으로 실행할 수 있는 방안을 제시하고 Javascript로 매핑하여 자동화가 가능함을 보였다. 스크립트 작성기를 만들면 Javascript를 작성하는 오버헤드를 줄일 수도 있다. 작성된 테스트 스크립트는 웹 기반 소프트웨어의 리그레션 테스트에 반복적으로 사용할 수 있다. As Internet has grown rapidly and been more complex by technology in connection with Web and requirement of business, qualify and reliability of Web application are getting important. It is necessary to study about testing method along with design technique specially in Web application. This paper explains automation method of user interface test to make test cases about user input form with HTML pages using by built-in browser objects. Examples shows the possibility of testing automation with Javascript objects get mapped. Overhead of writing Javascript can be reduced by making script generator. Generated test scripts are repeatedly used in regression testing Web-based application.