Adaptive Distinguishing Sequence Research Articles

Incomplete Adaptive Distinguishing Sequences for Non-Deterministic FSMs

Incomplete Adaptive Distinguishing Sequences for Non-Deterministic FSMs

State identification for labeled transition systems with inputs and outputs

For Finite State Machines (FSMs) a rich testing theory has been developed to discover aspects of their behavior and ensure their correct functioning. Although this theory has been frequently used, e.g. to check conformance of protocol implementations, its applicability is limited by restrictions of FSMs, in which inputs and outputs alternate, and outputs are determined by the previous input and state. Labeled Transition Systems with inputs and outputs (LTSs), as studied in ioco testing theory, provide a richer framework for testing component oriented systems, but lack the algorithms for test generation from FSM theory. In this article, we propose an algorithm for the fundamental problem of state identification during testing of LTSs. Our algorithm is a direct generalization of the well-known algorithm for computing adaptive distinguishing sequences for FSMs proposed by Lee and Yannakakis. Our algorithm has to deal with so-called compatible states, states that cannot be distinguished. Analogous to the result of Lee and Yannakakis, we prove that if an adaptive test exists that distinguishes all pairs of (incompatible) states of an LTS, our algorithm will find one. In practice, such perfect adaptive tests typically do not exist. However, in experiments with an implementation of our algorithm on a collection of (both academic and industrial) benchmarks, we find that that the adaptive tests produced by our algorithm still distinguish at least 99% of the incompatible state pairs.

State identification sequences from the splitting tree

Context: Software testing based on finite-state machines. Objective:Improving the performance of existing testing methods by construction of more efficient separating sequences, so that states entered by a system under test can be identified in a much shorter span of time. Method: This paper proposes an efficient way to construct separating sequences for subsets of states for any deterministic finite-state machine. It extends an existing algorithm that builds an adaptive distinguishing sequence (ADS) from a splitting tree to machines that do not possess an ADS. Our extension to this construction algorithm allows one not only to construct a separating sequence for any subset of states but also form sets of separating sequences, such as harmonized state identifiers (HSI) and incomplete adaptive distinguishing sequences, that are used by efficient testing and learning algorithms. Results: The experiments confirm that the length and number of test sequences produced by testing methods that use HSIs constructed by our extension is significantly improved. Conclusion:By constructing more efficient separating sequences the performance of existing test methods significantly improves.

Adaptive distinguishing test cases of nondeterministic finite state machines: test case derivation and length estimation

Abstract A top-down approach is presented for checking the existence and derivation of an adaptive distinguishing test case (called also an adaptive distinguishing sequence) for a nondeterministic finite state machine (NDFSM). When such a test case exists, the method returns a canonical test case that includes all other distinguishing tests of the given complete observable NDFSM. In the second part of the paper, a constructive approach is provided for deriving a class of complete observable NDFSMs with n states, n > 2, and 2 n − n − 1 inputs such that a shortest adaptive distinguishing test case for each NDFSM in the intended class has the length (height) 2 n − n − 1. In other words, we prove the reachability of the exponential upper bound on the length of a shortest adaptive distinguishing sequence for complete observable NDFSMs while for deterministic machines the upper bound is polynomial with respect to the number of states. For constructing the intended class of NDFSMs for a given n , we propose a special linear order over all the non-empty subsets without singletons of an n -element set. The obtained tight exponential upper bound initiates further research on identifying certain NDFSM classes where this upper bound is not reachable.

К синтезу адаптивных различающих последовательностей для конечных автоматов

К синтезу адаптивных различающих последовательностей для конечных автоматов

Distinguishing Sequences for Distributed Testing: Preset Distinguishing Sequences

This paper concerns the problem of testing from a finite state machine (FSM) $M$ modelling a system that interacts with its environment at multiple physically distributed interfaces, called ports. We assume that the distributed test architecture is used: there is a local tester at each port, the tester at port $p$ only observes events at $p$ and the testers do not interact during testing. This paper formalizes the notion of an adaptive test strategy and what it means for an adaptive test strategy to be controllable. We provide algorithms to check whether a global strategy is controllable and to generate a controllable adaptive distinguishing sequence (ADS). We prove that controllable ADS existence is PSPACE-Hard and that the problem of deciding whether $M$ has a controllable ADS with length $\ell $ is NP-Hard. In practice, there is likely to be a polynomial upper bound on the length of ADS in which we are interested and for this case the decision problem is NP-Complete.

Effective algorithms for constructing minimum cost adaptive distinguishing sequences

Context: Given a Finite State Machine (FSM), a checking sequence is a test sequence that determines whether the system under test is correct as long as certain standard assumptions hold. Many checking sequence generation methods use an adaptive distinguishing sequence (ADS), which is an experiment that distinguishes the states of the specification machine. Furthermore, it has been shown that the use of shorter ADSs yields shorter checking sequences. It is also known, on the other hand, that constructing a minimum cost ADS is an NP-hard problem and it is NP-hard to approximate. This motivates studying and investigating effective ADS construction methods.Objective:The main objective of this paper is to suggest new methods that can compute compact ADSs to be used in the construction of checking sequences.Method:We briefly present the existing ADS construction algorithms. We then propose generalizations of these approaches with a set of heuristics. We also conduct experiments to compare the size of the resultant ADSs and the length of the checking sequences constructed using these ADSs.Results:The results indicate that when the ADSs are constructed with the proposed methods, the length of the checking sequences may reduce up to 54% (40% on the average).Conclusions:In this paper, we present the state of the art ADS construction methods for FSMs and we propose generalizations of these methods. We show that our methods are effective in terms of computation time and ADS quality.

Distinguishing Sequences for Distributed Testing: Adaptive Distinguishing Sequences

This paper concerns the problem of testing from a finite state machine (FSM) | $M$ | modelling a system that interacts with its environment at multiple physically distributed interfaces, called ports. We assume that the distributed test architecture is used: there is a local tester at each port, the tester at port | $p$ | only observes events at | $p$ | and the testers do not interact during testing. This paper formalizes the notion of an adaptive test strategy and what it means for an adaptive test strategy to be controllable. We provide algorithms to check whether a global strategy is controllable and to generate a controllable adaptive distinguishing sequence (ADS). We prove that controllable ADS existence is PSPACE-Hard and that the problem of deciding whether | $M$ | has a controllable ADS with length | $\ell $ | is NP-Hard . In practice, there is likely to be a polynomial upper bound on the length of ADS in which we are interested and for this case the decision problem is NP-Complete .

Incomplete Distinguishing Sequences for Finite State Machines

Given a Finite State Machine (FSM) M , a Distinguishing Sequence (DS) is a test that identifies the state of M . While there are two types of DSs, preset DSs (PDSs) and adaptive DSs (ADSs), not all FSMs possess a DS. In this paper, we examine the problem of finding incomplete PDSs and ADSs, exploring associated optimisation problems: finding a largest set of states that has a DS and finding a smallest set of DSs that, between them, distinguish all of the states. We also propose a greedy algorithm to produce a small set of incomplete ADSs and use experiments to compare this with two previously published algorithms for generating state identifiers. We show that the optimisation problems related to incomplete ADSs and PDSs are PSPACE-complete as are corresponding approximation problems. In the experiments we found that incomplete ADSs produced by the proposed greedy algorithm led to relatively compact state identifiers.

Generalizing the DS-Methods for Testing Non-Deterministic FSMs

There exists a significant body of work devoted to so-called complete tests which guarantee the detection of all the faults in a given fault domain. Several methods for generating complete tests for finite state machines (FSMs) which are based on a distinguishing sequence (DS) have been proposed. These methods even if extended to use adaptive DSs apply only to deterministic FSMs and the question arises whether they can be extended to non-deterministic FSMs to test for trace inclusion. In this paper, we generalize the notion of DS to a so-called total state separator, which is an adaptive experiment distinguishing states in any FSM that is trace included into the specification FSM. We then propose a method to test non-deterministic FSMs for trace inclusion. State separator is a key means of the proposed method, which has two phases: in the first phase, a preset test is constructed, which should be repeatedly applied to a non-deterministic implementation, thus requiring its resetting; in the second phase, the implementation is tested online and no reset is required. To the best of our knowledge, this is the first method which tests non-deterministic FSMs for the trace inclusion conformance relation, while avoiding resetting implementations to re-execute tests for transition verification.

Hardness and inapproximability of minimizing adaptive distinguishing sequences

An adaptive distinguishing sequence (ADS) can be used for identifying an unknown initial state of a finite state machine (FSM). It has been long known that checking the existence of an ADS for an FSM, and finding an ADS for an FSM when one exists, can be performed in polynomial time. However, the problem of finding a minimum ADS has not been studied so far. Generating a minimum ADS is especially motivated when such an ADS is used repeatedly, e.g. for the construction of a test sequence. We introduce a number of metrics to define a minimum ADS and show that the problem of generating a minimum ADS with respect to these metrics is NP-complete. In addition, we provide inapproximability results for these hard problems and show that not only deciding but also approximating such a minimum ADS is a hard problem. We modify the only polynomial time ADS generation algorithm existing, and experimentally show that these modifications construct reduced ADSs. We also validate the motivation of ADS minimization by presenting experimental results on the effect of using reduced ADSs to generate test sequences.

Синтез проверяющих последовательностей для недетерминированных автоматов относительно редукции

Most FSM based methods for test derivation are developed for initialized Finite State Machines (FSM) and the latter means that a reliable reset is assumed in an implementation under test in order to glue test sequences together. If the reset is rather expensive then the number of test sequences has to be reduced and when it is reduced to a single sequence, this sequence is called a checking sequence. In this paper, a methods is proposed for deriving an adaptive checking sequence when the specification FSM is nondeterministic and the conformance relation is the reduction relation. The latter means that the behavior of a conforming implementation should be contained in the behavior of the specification. A method returns an adaptive checking sequence that detects each nonconforming implementation that has not more states than the specification FSM under the conditions that the specification has a distinguishing sequence and a deterministic strongly connected submachine. These conditions can be weakened for the case when the specification has a distinguishing test case and each state of the specification is definitely reachable from another state. The testing process is adaptive, i.e., the next input is determined based on the outputs produced for the previous inputs. Such adaptive distinguishing sequences can be shorter than preset checking sequences.

Testing finite-state machines: state identification and verification

We study the complexity of two fundamental problems in the testing of finite-state machines. 1) Distinguishing sequences (state identification). We show that it is PSPACE-complete to determine whether a finite-state machine has a preset distinguishing sequence. There are machines that have distinguishing sequences, but only of exponential length. We give a polynomial time algorithm that determines whether a finite-state machine has an adaptive distinguishing sequence. (The previous classical algorithms take exponential time.) Furthermore, if there is an adaptive distinguishing sequence, then we give an efficient algorithm that constructs such a sequence of length at most n(n/spl minus/1)/2 (which is the best possible), where n is the number of states. 2) Unique input output sequences (state verification). It is PSPACE-complete to determine whether a state of a machine has a unique input output sequence. There are machines whose states have unique input output sequences but only of exponential length.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>