Input-output Transition Systems Research Articles

Clustering of Web Application and Testing of Asynchronous Communication

In model-based testing, testing is done using Input Output Transition System (IOTS) or Finite State Machine (FSM). In synchronous testing, it is easy to know the proper sequence of input output of the system under testing, whereas, in asynchronous communication, it is tough to trace the sequences of inputs and outputs. A web application is an example of asynchronous processing, in which input/output queue management is a common issue for testing in asynchronous JavaScript and XML user interfaces. This article discusses asynchronous testing of web application by taking a Gmail account as a case study. In this article, the web application is partitioned into clusters by determining logical web pages that implement logical functions. The clusters of page objects are represented using finite state machines. From the FSMs, test scenarios are generated. The size of the test cases covering longer paths is decreased by making clusters of states. The authors conduct a queue analysis using a test architecture to investigate the behavior of the system under testing. Finally, the test cases are generated using the Selenium tool.

Formal mutation testing for Circus

Context: The demand from industry for more dependable and scalable test-development mechanisms has fostered the use of formal models to guide the generation of tests. Despite many advancements having been obtained with state-based models, such as Finite State Machines (FSMs) and Input/Output Transition Systems (IOTSs), more advanced formalisms are required to specify large, state-rich, concurrent systems. Circus, a state-rich process algebra combining Z, CSP and a refinement calculus, is suitable for this; however, deriving tests from such models is accordingly more challenging. Recently, a testing theory has been stated for Circus, allowing the verification of process refinement based on exhaustive test sets.Objective: We investigate fault-based testing for refinement from Circus specifications using mutation. We seek the benefits of such techniques in test-set quality assertion and fault-based test-case selection. We target results relevant not only for Circus, but to any process algebra for refinement that combines CSP with a data language.Method: We present a formal definition for fault-based test sets, extending the Circus testing theory, and an extensive study of mutation operators for Circus. Using these results, we propose an approach to generate tests to kill mutants. Finally, we explain how prototype tool support can be obtained with the implementation of a mutant generator, a translator from Circus to CSP, and a refinement checker for CSP, and with a more sophisticated chain of tools that support the use of symbolic tests.Results: We formally characterise mutation testing for Circus, defining the exhaustive test sets that can kill a given mutant. We also provide a technique to select tests from these sets based on specification traces of the mutants. Finally, we present mutation operators that consider faults related to both reactive and data manipulation behaviour. Altogether, we define a new fault-based test-generation technique for Circus.Conclusion: We conclude that mutation testing for Circus can truly aid making test generation from state-rich model more tractable, by focussing on particular faults.

Generation of complete test suites from mealy input/output transition systems

AbstractModel-based testing overcomes challenges in software testing by generating automated test cases from behavior models, e.g. finite state machine (FSM) and input/output transition system (IOTS). Despite the existence of methods for IOTSs, the problem of selection of test cases is an important and difficult topic. The current methods from IOTSs do not provide the same support offered by the existing theory for FSMs, as complete fault coverage. In this paper, we propose a test generation method for IOTSs based on the W method developed for FSMs. The basic idea is to generate a transition cover set and a characterization set and concatenate them to generate complete test suites for IOTSs in a bounded number of steps. The method generates test suites with complete fault coverage for a given fault domain and is targeted at a class of IOTSs, called mealy IOTS, which accepts inputs only in stable states. Results from a case study show the proposed algorithm can achieve better results than a classical method for IOTSs.

A More Precise Implementation Relation for Distributed Testing

There has been significant interest in distributed testing from an input output transition system. Previous work introduced an implementation relation dioco that was defined in terms of an equivalence relation on traces (sequences of observations). This paper considers an alternative approach in which an observation made in testing is a tuple of local traces, one for each tester. This paper defines such an implementation relation diocoo in terms of the possible observations regarding the system under test and the specification. It shows that diocoo is strictly weaker than dioco but is equivalent to dioco if processes cannot be output-divergent. Interestingly, this shows that the previous definition of dioco is too strong for output-divergent processes. We also prove that the Oracle problem is NP-complete but can be solved in polynomial time if there is an upper bound on the number of local testers.

Diagnosability Analysis of Input/Output Discrete-Event Systems Using Model-Checking

Abstract: This paper deals with analysis of diagnosability and K -diagnosability of dynamic systems in a model-checking framework. Dynamic systems are abstracted here as Discrete-Event Systems (DESs) and modeled by Input/Output Transition Systems (IOTSs). We reformulate diagnosability issues using CTL formula while considering extended definitions of diagnosability. Moreover, we introduce a formal definition of K -diagnosability in model-checking framework and we discuss the problem of K min -diagnosability (the minimal value of K ensuring diagnosability). We also show how diagnosability analysis in model-checking framework can be extended in order to deal with repeated/intermittent failures. In this regard, the case of [1-∞]-diagnosability analysis is investigated. Finally, some of these theoretical contributions are illustrated through a benchmark.

The complexity of asynchronous model based testing

In model based testing (MBT), testing is based on a model M that typically is expressed using a state-based language such as an input output transition system (IOTS). Most approaches to MBT assume that communications between the system under test (SUT) and its environment are synchronous. However, many systems interact with their environment through asynchronous channels and the presence of such channels changes the nature of testing. In this paper we investigate the situation in which the SUT interacts with its environment through asynchronous channels and the problems of producing test cases to reach a state, execute a transition, or to distinguish two states. In addition, we investigate the Oracle Problem. All four problems are explored for both FIFO and non-FIFO channels. It is known that the Oracle Problem can be solved in polynomial time for FIFO channels but we also show that the three test case generation problems can also be solved in polynomial time in the case where the IOTS is observable but the general test generation problems are EXPTIME-hard. For non-FIFO channels we prove that all of the test case generation problems are EXPTIME-hard and the Oracle Problem in NP-hard, even if we restrict attention to deterministic IOTSs.

Talking quiescence: a rigorous theory that supports parallel composition, action hiding and determinisation

The notion of quiescence - the absence of outputs - is vital in both behavioural modelling and testing theory. Although the need for quiescence was already recognised in the 90s, it has only been treated as a second-class citizen thus far. This paper moves quiescence into the foreground and introduces the notion of quiescent transition systems (QTSs): an extension of regular input-output transition systems (IOTSs) in which quiescence is represented explicitly, via quiescent transitions. Four carefully crafted rules on the use of quiescent transitions ensure that our QTSs naturally capture quiescent behaviour. We present the building blocks for a comprehensive theory on QTSs supporting parallel composition, action hiding and determinisation. In particular, we prove that these operations preserve all the aforementioned rules. Additionally, we provide a way to transform existing IOTSs into QTSs, allowing even IOTSs as input that already contain some quiescent transitions. As an important application, we show how our QTS framework simplifies the fundamental model-based testing theory formalised around ioco.

Overcoming controllability problems in distributed testing from an input output transition system

This paper concerns the testing of a system with physically distributed interfaces, called ports, at which it interacts with its environment. We place a tester at each port and the tester at port p observes events at p only. This can lead to controllability problems, where the observations made by the tester at a port p are not sufficient for it to be able to know when to send an input. It is known that there are test objectives, such as executing a particular transition, that cannot be achieved if we restrict attention to test cases that have no controllability problems. This has led to interest in schemes where the testers at the individual ports send coordination messages to one another through an external communications network in order to overcome controllability problems. However, such approaches have largely been studied in the context of testing from a deterministic finite state machine. This paper investigates the use of coordination messages to overcome controllability problems when testing from an input output transition system and gives an algorithm for introducing sufficient messages. It also proves that the problem of minimising the number of coordination messages used is NP-hard.

Implementation relations and test generation for systems with distributed interfaces

Some systems interact with their environment at physically distributed interfaces called ports and we separately observe sequences of inputs and outputs at each port. As a result we cannot reconstruct the global sequence that occurred and this reduces our ability to distinguish different systems in testing or in use. In this paper we explore notions of conformance for an input output transition system that has multiple ports, adapting the widely used ioco implementation relation to this situation. We consider two different scenarios. In the first scenario the agents at the different ports are entirely independent. Alternatively, it may be feasible for some external agent to receive information from more than one of the agents at the ports of the system, these local behaviours potentially being brought together and here we require a stronger implementation relation. We define implementation relations for these scenarios and prove that in the case of a single-port system the new implementation relations are equivalent to ioco. In addition, we define what it means for a test case to be controllable and give an algorithm that decides whether this condition holds. We give a test generation algorithm to produce sound and complete test suites. Finally, we study two implementation relations to deal with partially specified systems.

Generating asynchronous test cases from test purposes

Context Input/output transition system (IOTS) models are commonly used when next input can arrive even before outputs are produced. The interaction between the tester and an implementation under test (IUT) is usually assumed to be synchronous. However, as the IUT can produce outputs at any moment, the tester should be prepared to accept all outputs from the IUT, or else be able to block (refuse) outputs of the implementation. Testing distributed, remote applications under the assumptions that communication is synchronous and actions can be blocked is unrealistic, since synchronous communication for such applications can only be achieved if special protocols are used. In this context, asynchronous tests can be more appropriate, reflecting the underlying test architecture which includes queues. Objective In this paper, we investigate the problem of constructing test cases for given test purposes and specification input/output transition systems, when the communication between the tester and the implementation under test is assumed to be asynchronous, performed via multiple queues. Method When issuing verdicts, asynchronous tests should take into account a distortion caused by the queues in the observed interactions. First, we investigate how the test purpose can be transformed to account for this distortion when there are a single input queue and a single output queue. Then, we consider a more general problem, when there may be multiple queues. Results We propose an algorithm which constructs a sound test case, by transforming the test purpose prior to composing it with the specification without queues. Conclusion The proposed algorithm mitigates the state explosion problem which usually occurs when queues are directly involved in the composition. Experimental results confirm the resulting state space reduction.