Unique Input Output Sequences Research Articles

$\mathcal k$-branching uio sequences for partially specified observable non-deterministic fsms

In black-box testing, test sequences may be constructed from systems modelled as deterministic finite-state machines (DFSMs) or, more generally, observable non-deterministic finite state machines (ONFSMs). Test sequences usually contain state identification sequences, with unique input output sequences ( <small>UIOs</small> ) often being used with DFSMs. This paper extends the notion of <small>UIOs</small> to ONFSMs. One challenge is that, as a result of non-determinism, the application of an input sequence can lead to exponentially many expected output sequences. To address this scalability problem, we introduce <inline-formula><tex-math notation="LaTeX">${\mathcal K}$</tex-math></inline-formula> - <small>UIOs</small> : <small>UIOs</small> that lead to at most <inline-formula><tex-math notation="LaTeX">${\mathcal K}$</tex-math></inline-formula> output sequences from states of <inline-formula><tex-math notation="LaTeX">$M$</tex-math></inline-formula> . We show that checking <inline-formula><tex-math notation="LaTeX">${\mathcal K}$</tex-math></inline-formula> - <small>UIO</small> existence is PSPACE-Complete if the problem is suitably bounded; otherwise it is in EXPSPACE and PSPACE-Hard. We provide a massively parallel algorithm for constructing <inline-formula><tex-math notation="LaTeX">${\mathcal K}$</tex-math></inline-formula> - <small>UIOs</small> and the results of experiments on randomly generated and real FSM specifications. The proposed algorithm was able to construct <small>UIOs</small> in cases where the existing <small>UIO</small> generation algorithm could not and was able to construct <small>UIOs</small> from FSMs with 38K states and 400K transitions.

Oracles for Testing Software Timeliness with Uncertainty

Uncertainty in timing properties (e.g., detection time of external events) is a common occurrence in embedded software systems, since these systems interact with complex physical environments. Such time uncertainty leads to non-determinism. For example, time-triggered operations may either generate different valid outputs across different executions or experience failures (e.g., results not being generated in the expected time window) that occur only occasionally over many executions. For these reasons, time uncertainty makes the generation of effective test oracles for timing requirements a challenging task. To address the above challenge, we propose Stochastic Testing with Unique Input Output Sequences, an approach for the automated generation of stochastic oracles that verify the capability of a software system to fulfill timing constraints in the presence of time uncertainty. Such stochastic oracles entail the statistical analysis of repeated test case executions based on test output probabilities predicted by means of statistical model checking. Results from two industrial case studies in the automotive domain demonstrate that this approach improves the fault detection effectiveness of tests suites derived from timed automata compared to traditional approaches.

Parallel Algorithms for Testing Finite State Machines:Generating UIO Sequences

This paper describes an efficient parallel algorithm that uses many-core GPUs for automatically deriving Unique Input Output sequences (UIOs) from Finite State Machines. The proposed algorithm uses the global scope of the GPU's global memory through coalesced memory access and minimises the transfer between CPU and GPU memory. The results of experiments indicate that the proposed method yields considerably better results compared to a single core UIO construction algorithm. Our algorithm is scalable and when multiple GPUs are added into the system the approach can handle FSMs whose size is larger than the memory available on a single GPU.

A hybrid optimization approach to conformance testing of finite automata

Unique Input–Output sequences (UIOs) are quite commonly used in conformance testing. Unfortunately finding UIOs of minimal length is an NP hard problem. This study presents a hybrid approach to generate UIOs automatically on a basis of the finite state machine (FSM) specification. The proposed hybrid approach harnesses the benefits of hill climbing (Greedy search) and heuristic algorithm. Hill climbing, which exploits domain knowledge, is capable of quickly generating good result however it may get stuck in local minimum. To overcome the problem we used a set of parameters called the seed, which allows the algorithm to generate different results for a different seed. The hill climbing generates solutions implied by the seed while the Genetic Algorithm is used as the seed generator. We compared the hybrid approach with Genetic Algorithm, Simulated Annealing, Greedy Algorithm, and Random Search. The experimental evaluation shows that the proposed hybrid approach outperforms other methods. More specifically, we showed that Genetic Algorithm and Simulated Annealing exhibit similar performance while both of them outperform Greedy Algorithm. Finally, we generalize the proposed hybrid approach to seed-driven hybrid architectures and elaborate on how it can be adopted to a broad range of optimization problems.

A study on the extended unique input/output sequence

The unique input/output (UIO) sequence is an important state identification technique in the FSM-based protocol conformance testing. However, some states of a FSM might have no UIO sequence. To address the above problem, this paper introduces an extended UIO sequence, called a GUIO sequence, which distinguishes a designated state from the remaining states by a group-by-group means. The problem of searching the optimal GUIO sequence is NP-hard. In this paper, we present a GUIO search method based on a greedy heuristics and a φ-hop search approach. The proposed method can obtain desirable GUIO sequences with relatively low computational complexity.

Guest Editorial: special issue on evolutionary optimisation and learning

Problem-solving techniques based on simulated evolution, particularly those inspired by the Darwinian theory of natural selection and biological evolution, have attracted great attention. These evolutionary optimisation and learning algorithms differ from the traditional optimisation and learning methods in that they involve a search from a population of solutions. Over the past decades, evolutionary optimisation and learning algorithms have been successfully applied to a wide variety of real-world problems. This special issue brings together some recent works from a wide range of topics concerning evolutionary optimisation and learning, including Differential Evolution, Cellular Genetic Algorithms, Particle Swarm Optimisation, Artificial Immune System, Coevolution, Crossover Analysis, Dynamic Combinatorial Optimisation, and Multimodal Optimisation. The eight papers included in this special issue originated from SEAL’08 (7th International Conference on Simulated Evolution and Learning), but have been substantially extended and revised from the conference version. These further extended papers were again rigorously reviewed in two rounds by at least three anonymous reviewers. The paper by Lehre and Yao investigates the impact of the parameter settings particularly the acceptance criterion in evolutionary algorithms (EAs) and the crossover operator SSGA when computing unique input output sequences (UIOs) from finite state machines. The objective is to identify simple, archetypical cases where these EA parameter settings have a particularly strong effect on the runtime of the algorithm. This result shows that minor modification in evolutionary algorithms can have an exponentially large runtime impact for computing UIOs and that when computing UIOs, the crossover operator can be essential, and simple EAs can be inefficient. The paper by Ronkknen et al. describes a software framework for generating multimodal test functions. The framework provides an easy way to construct parameterisable functions and offers an environment for testing multimodal optimisation algorithms. A number of function families with different characteristics are included. The framework implements new parameterisable function families for generating desired landscapes. In addition, the framework implements a selection of well-known test functions from the literature, which can be rotated and stretched. The software module can be easily imported into any optimisation algorithm implementation compatible with the C-programming language. As an example, eight optimisation approaches are compared by their ability to locate several global optima over a set of 16 functions with different properties generated by the proposed module. The paper by Dick and Whigham introduces two modifications to the local sharing method for multimodal optimisation. The first alters local sharing so that parent selection and fitness sharing operate at two different spatial levels: parent selection is performed within small demes, M. Zhang (&) School of Engineering and Computer Science, Victoria University of Wellington, PO Box 600, Wellington, New Zealand e-mail: mengjie.zhang@ecs.vuw.ac.nz

Software testing and quality assurance: theory and practice

Software testing and quality assurance: theory and practice

Evolutionary generation of unique input/output sequences for class behavioral testing

Object-oriented software is composed of classes. Their behaviors are usually modeled with state diagrams or finite state machines (FSMs). Testing classes is regarded as testing FSMs in which unique input/output (UIO) sequences are widely applied. The generation of UIO sequences is shown to be an undecidable problem. For these problems, genetic algorithms (GAs) may offer much promise. This paper reports some primary results of on-going research on evolutionary testing classes. First, we explain how to define UIO sequence generation as a search problem, and then describe adapting genetic algorithms to generating UIO sequences. Special issues of using genetic algorithms such as solution representation, validity checking and fitness definition are discussed in detail. Primary experiments confirm the applicability and feasibility of applying GAs to UIO sequence generation.

Synthesis of finite state machines for improved state verification

Finite State Machines (FSMs) are used in diverse areas to model hardware and software systems. Verification of FSMs is essential to ensure reliability of systems. To verify that a machine is in an expected state in testing, Unique Input/Output (UIO) sequences are used. The aforementioned testing methodology requires that each state in the FSM has an UIO. However, it is possible for a given machine that few or even none of its states have an UIO sequence. This paper presents a guided heuristic algorithm for synthesizing FSMs such that each state has an UIO sequence. The states of an FSM with identical I/O labels on transitions are grouped in order to identify the states which do not possess UIO sequence. The transitions are then augmented by adding extra output terminals incrementally so that new UIO sequences are created for the states. A greedy approach is used to optimize the number of added outputs. Initially, the transitions which lead to state convergence (i.e., transitions with identical input/output labels taking a set of states to the same next state) and constrained self-loop (i.e., transitions taking a set of states either to itself or leads to state convergence) are identified since a state with only these transitions will never have a UIO sequence. Extra output terminals are added to the FSM which are used only while testing and the augmented output labels make sure that the states are neither convergent nor has constrained self-loop, thereby ensuring UIO sequence. The proposed algorithm, referred to as AUGP, was tested with a large number of FSMs including the Microelectronics Center of North Carolina (MCNC) FSM benchmarks. The augmented state transition table was used as input to a UIO computation algorithm (developed by the same authors [Ahmad I, et al. IEE Proc Comput Digital Tech 2004;151(2):131]) to check the performance of the augmentation algorithm and the tested FSMs were found to possess UIO sequence for all states.

Automated Unique Input Output Sequence Generation for Conformance Testing of FSMs

This paper describes a method for automatically generating unique input output (UIO) sequences for FSM conformance testing. UIOs are used in conformance testing to verify the end state of a transition sequence. UIO sequence generation is represented as a search problem and genetic algorithms are used to search this space. Empirical evidence indicates that the proposed method yields considerably better (up to 62% better) results compared with random UIO sequence generation.

LANG – algorithm for constructing unique input/output sequences in finite-state machines

Finite-state-machines (FSMs) model a variety of hardware and software systems. Unique input/output (UIO) sequences are used in generation of test sequences to verify that a machine is in an expected state, which in turn ensures system reliability. The paper presents an efficient algorithm for computing UIO sequences for completely and incompletely specified FSMs with binary inputs. Set of states which produces identical outputs for nonconflicting inputs are partitioned by incrementally constructing a successor tree. The transitions of FSM are modified to create a set of more specified transitions. This also resulted in constructing UIO sequences for an additional number of states. By a heuristic breadth-first search of successor tree, shorter test sequences are generated. In addition, classifying the nodes as ‘active’, ‘inactive’ and ‘dead’ and application of pruning rules and termination checks applied while partitioning the states reduced the search space and speeded up UIO-sequence construction. A chain-node search technique was used when successor tree failed to construct UIO sequences in some FSMs. The proposed algorithm, designated LANG, was tested with a large number of FSMs including the MCNC FSM benchmarks. To show the effectiveness of the proposed algorithm, results are compared with existing UIO-sequence-computation algorithms. The proposed algorithm computed shorter UIO sequences in negligible CPU time.

Test generation for sequential circuits using state transition diagram and test generation for combinatorial circuit part

AbstractThis study deals with single stuck‐at faults in sequential circuits; in particular, a test generation method featuring low generation complexity and short test sequences. With the proposed method, short test sequences are generated using shortest local connections between activation vector set found for combinatorial circuit part at gate level, and state transitions found from state transition diagram at functional level. Generated test sequences are input sequences composed of subsequences referred to as single test sequences. Single test sequences are composed of state control sequences that control activation vectors and their states, and UIO (Unique Input/Output) sequences that confirm states after applying activation vectors. Test sequences are generated so as to locally minimize total length of state control sequences and UIO sequences in connected single test sequences, which results in shorter test sequences as compared to previous generation methods. Evaluation experiments using MCNC benchmarks proved that test sequences generated by the proposed method were shorter by up to 28.8% as compared to previous methods while fault coverage was above 98% on the average. © 2001 Scripta Technica, Electron Comm Jpn Pt 2, 84(8): 20–28, 2001

An extended-UIO-based method for protocol conformance testing

Verification of protocols is performed through conformance testing. The aim of this paper is to introduce the conformance test generation approach for protocols described by means of Finite State Machines. A functional fault model is adopted and the state discrimination is performed by applying an extended version of Unique Input Output Sequences (UIO), which, different from classical UIO, can always be found in any state. Both algorithms for efficient extended UIO sequence identification and for optimal test sequence generation are presented, together with the experimental results on different protocol descriptions.

Automated Test Generation from Specifications Based on Formal Description Techniques

This paper describes a research result on automatic generation of abstract test cases from formal specifications by applying many related algorithms and techniques such as the testing framework, rural Chinese postman tour and unique input output sequence concepts. In addition, an efficient algorithm for verifying the strong connectivity of the reference finite state machine and the concept of unique event sequence are explained. We made use of several techniques to form an integrated framework for abstract test case generation from LOTOS and SDL specifications. A prototype of the proposed framework has been built with special attention to real protocol in order to generate the executable test cases in an automatic way. The abstract test cases in tree and tabular combined notation (TTCN) language will be applied to the TTCN compiler in order to obtain the executable test cases which are relevant to the industrial application.

Efficient computation of unique input/output sequences in finite state machines

This paper makes two contributions toward computing unique input/output (UIO) sequences in finite-state machines. Our first contribution is to compute all UIO sequences of minimal lengths in a finite-state machine. Our second contribution is to present a generally efficient algorithm to compute a UIO sequence for each state, if it exists. We begin by defining a path vector, vector perturbation, and UIO tree. The perturbation process allows us to construct the complete UIO tree for a machine. Each sequence of input/output from the initial vector of a UIO tree to a singleton vector represents a UIO sequence. Next, we define the idea of an inference rule that allows us to infer UIO sequences of a number of states from the UIO sequence of some state. That is, for a large class of machines, it is possible to compute UIO sequences for all possible states from a small set of initial UIOs. We give a modified depth-first algorithm, called the hybrid approach, that computes a partial UIO tree, called an essential subtree, from which UIO sequences of all possible states can be inferred. Using the concept of projection machines, we show that sometimes it is unnecessary to construct even a partial subtree. We prove that if a machine remains strongly connected after deleting all the converging transitions, then all of the states have UIO sequences. To demonstrate the effectiveness of our approach, we develop a tool to perform experiments using both small and large machines.

Improving the UIOv-method for protocol conformance testing

The UIOv-method for testing the conformance of a protocol implementation to its specification is improved to generate a shorter test sequence. The traditional UIOv-method utilizes minimum-length Unique Input/Output (UIO) sequences to verify the implementation's states, and thus suffers from a long verification part for checking that these UIO sequences are uniquely implemented. The improved method removes this constraint so that each state can select its UIO sequence from a large set of UIO sequences, and thus some choices may shorten the verification. The exclusive tree data structure is also proposed to represent the mutually exclusive relation between UIO sequences (two UIO sequences are called mutually exclusive if no verification is needed within them); each exclusive tree corresponds to a set of UIO sequences. An algorithm is proposed to generate the exclusive tree whose corresponding UIO sequences can be used with the UIOv-method to shorten the verification while not lengthening the total test sequence. Experimentation of two protocol examples and the Q.931 network-side protocol indicate a length reduction for the verification of 100%, 43% and 64%, respectively, and for the total test sequence of 13.8%, 8.6% and 12.7%, respectively.

Testing finite-state machines: state identification and verification

We study the complexity of two fundamental problems in the testing of finite-state machines. 1) Distinguishing sequences (state identification). We show that it is PSPACE-complete to determine whether a finite-state machine has a preset distinguishing sequence. There are machines that have distinguishing sequences, but only of exponential length. We give a polynomial time algorithm that determines whether a finite-state machine has an adaptive distinguishing sequence. (The previous classical algorithms take exponential time.) Furthermore, if there is an adaptive distinguishing sequence, then we give an efficient algorithm that constructs such a sequence of length at most n(n/spl minus/1)/2 (which is the best possible), where n is the number of states. 2) Unique input output sequences (state verification). It is PSPACE-complete to determine whether a state of a machine has a unique input output sequence. There are machines whose states have unique input output sequences but only of exponential length.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Protocol conformance testing using multiple UIO sequences

Automatic generation of conformance test sequences for communication protocols by means of unique input/output (UIO) sequences is addressed. It is shown that if multiple minimum-length UIO sequences are computed for each state of the finite-state-machine (FSM) specification, then the length of the resulting test sequence is significantly reduced without an appreciable increase in the time needed to compute the sequence. An algorithm for assignment of the multiple UIO sequences is given. This algorithm, which is based on network flow, is polynomial in the number of states and transitions of the FSM and is effective in reducing the overall length of the test sequence. >

A protocol test generation procedure

A procedure presented here generates test sequences for checking the conformity of an implementation to the control portion of a protocol specification, which is modeled as a deterministic finite-state machine (FSM). A test sequence generated by the procedure given here tours all state transitions and uses a unique signature for each state, called the Unique Input/Output (UIO) sequence. A UIO sequence for a state is an input/output behavior that is not exhibited by any other state. An algorithm is presented for generating a minimum-length UIO sequence, should it exist, for a given state. UIO sequences may not exist for some states.

A new technique for generating protocol test

A novel procedure presented here generates test sequences for checking the conformity of protocol implementations to their specifications. The test sequences generated by this procedure only detect the presence of many faults, but they do not locate the faults. It can always detect the problem in an implementation with a single fault. A protocol entity is specified as a finite state machine (FSM). It typically has two interfaces: an interface with the user and with the lower-layer protocol. The inputs from both interfaces are merged into a single set I and the outputs from both interfaces are merged into a single set O. The implementation is assumed to be a black box. The key idea in this procedure is to tour all states and state transitions and to check a unique signature for each state, called the Unique Input/Output (UIO) sequence. A UIO sequence for a state is an I/O behavior that is not exhibited by any other state.