Protocol Conformance Testing Research Articles

A reduced test suite for protocol conformance testing

A reduced test suite for protocol conformance testing

Structural analysis of protocol specifications and generation of maximal fault coverage conformance test sequences

A theoretical analysis of the fault coverage of conformance test sequences for communication protocols specified as finite state machines is presented. Faults of different types are considered, and their effect on testing is analyzed. The interaction between faults of different categories and the impact it has on conformance testing is investigated. Fault coverage is defined for the testing of both incompletely-specified machines (ISMs) and completely-specified machines (CSMs). An algorithm is presented to generate test sequences with maximal fault coverage for the testing of ISMs. It is then augmented for the testing of CSMs, and finally a technique is presented for generating test sequences which provides guaranteed maximal fault coverage for the conformance testing of communication protocols.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Active tester based conformance testing strategy

A tester has been developed for the conformance testing of communication protocols. Protocol implementation can be checked on the basis of the standard test suite similarly to active testers but our version also applies a test strategy to handle the indeterministic behaviour of protocols by means of relations introduced between the test sequences of the test suite. A brief overview of the various existing testers is given prior to describing the starting point of our development and its theoretical background. The implementation of the tester is based on a three-level test architecture to realize the test strategy and it results in a flexible testing.

Extending ASN.1 into a full-fledged constraint language in the context of OSI protocol conformance testing

Upper layer (UL) protocols of the OSI model use complicated data structures defined with the aid of a Syntax Description Language called ASN.1. During Conformance Testing of OSI UL products, PDUs sent to and expected from the Implementation Under Test (IUT) must be described as constraints over ASN.1 types. ASN.1, however, being primarily designed as a Syntax and Encoding Definition Language, does not serve well the purpose of Constraint Specification. In this paper we present an extension to ASN.1, called Constraint Specification Notation One (CSN.1), which allows specification of arbitrarily complex constraints, as these are needed in Conformance Testing specifications.

Optimal length test sequence generation using distinguishing sequences

The optimization of the length of test sequences for finite state machine based protocol conformance testing is studied. The study focuses on test generation methods, called D-methods, that utilize distinguishing sequences in the construction of test segments. The extent of the optimization of the length of a test sequence is investigated with respect to two cases. The first case establishes the lower bound for the length of test sequences generated by any D-method that overlaps test segments. The second case establishes the lower bound for the length of test sequences generated by any D-method that does not overlap test segments. It is observed that the reduction in the length of test sequences due to overlapping is significant. An efficient algorithm for the generation of test sequences is proposed. This algorithm utilizes a distinguishing sequence and overlaps test segments. Sufficiency conditions are given both for finding a minimum- length test sequence in polynomial time and for constructing the optimal length test sequences by this algorithm. >

On the generation of minimal-length conformance tests for communication protocols

A technique for generating a test sequence for conformance testing of communication protocols is presented. This approach shows that it is possible to generate optimal-length test sequences which include multiple unique input/output (UIO) sequences and overlapping under certain conditions. In the absence of the above-mentioned conditions, a heuristic technique is used to obtain suboptimal solutions which show significant improvement over optimal solutions without overlapping. The technique is illustrated by the example of the NBS Class 4 Transport Protocol (TP4). The computational complexity of the algorithm is compared with that of previous techniques. A brief discussion of bounds on test sequence length is presented, and the results are compared with these bounds. >

Verification of protocol conformance test cases using reachability analysis

A methodology is presented to verify manually written test cases against the formal specification of a protocol. Initially, a protocol and a test case are modeled as nondeterministic finite state machines and test case verification is viewed as a reachability analysis problem. An existing reachability analysis algorithm, based on the well-known perturbation technique, is modified to take nondeterminism in protocols and special test case features (timeouts and OTHERWISE events) into account. Correctness aspects of the reachability algorithm are proved. The notion of a synchronization error manifesting in a test case due to the nondeterministic nature of a protocol specification is studied. To verify data flow aspects of test cases, we extend our technique by modeling the test case and protocol specification as extended finite state machines. A test case from a proprietary test suite for the transport protocol Class 2 is taken as an example and is shown to contain several design errors.

Finite state machine based formal methods in protocol conformance testing: from theory to implementation

Abstract Recent evolution of communication networks has led toward increasingly complex communication protocols to interconnect heterogeneous systems. In order for such systems to function properly, communication protocols require formal methodologies for verification, implementation and testing. In this paper, the use of formal methods for protocol conformance testing is presented. Four major formal techniques for conformance test generation reported in the literature are compared: transition tours, distinguishing sequences, characterizing sequences and unique input/output sequences. The implementation of each approach as a test generation methodology is illustrated. The result is the capability to generate abstract test suites that require a minimum number of steps to execute while maintaining complete coverage of the state transitions (or test purposes) of the protocol. The impact of the formal methodologies on the practical aspects of conformance testing, including the test system implementation and automatic test script generation, is discussed. The experience based on testing various protocol implementations from multiple manufacturers at AT&T Bell Laboratories is described. The protocols tested include X.25 and ISDN Q.921 and Q.931 for basic and primary rate interfaces. In order to standardize various efforts on conformance testing, the ISO and the CCITT have developed principles to specify abstract and executable test suites, and methodologies to design testbed architectures. The relationship between such standards and the formal methods for test generation is explored.

Experience with formal methods in protocol development

Communication protocols are complex software systems and key components of computer networks. They must be specified completely and implemented correctly. The development of a protocol typically involves several tedious steps. Experience has shown that the use of informal techniques in these steps of protocol development generally produces systems with errors and undesirable behaviors. Formal methods permit the development of highly reliable and easily maintainable communication protocols. This paper presents our experience with using formal methods in the specification, verification, implementation, and conformance testing of OSI protocols.

Test generation from LOTOS specifications

A novel approach is proposed for generating test cases from LOTOS specifications for protocol conformance testing. A set of transformation rules is applied to the specification, and the specification is mapped to an extended finite state machine called a chart. The flow of data in the chart is modeled by a data flow graph. Test cases are generated from the chart. Program testing strategies are applied for test case selection. A tool implementing this methodology on a workstation is also discussed. >

An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours

A method for generating test sequences for checking the conformance of a protocol implementation to its specification is described. A rural Chinese postman tour problem algorithm is used to determine a minimum-cost tour of the transition graph of a finite-state machine. It is shown that, when the unique input/output sequence (UIO) is used in place of the more cumbersome distinguishing sequence, both the controllability and observability problems of the protocol testing problem are addressed, providing an efficient method for computing a test sequence for protocol conformance testing.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An application of explanation-based learning to protocol conformance testing

A method that employs a machine learning technique for the semiautomatic generation of protocol-conformance test-sequence requirements is described. Given a protocol knowledge representation and some high-level nonexecutable descriptions of protocol behavior, a learning algorithm based on extended explanation-based generalization produces conformance test-sequence requirements for a protocol implementation under test. The role of learning is to compile relevant parts of protocol knowledge into behaviors, consequently inferring executable protocol behaviors. This inference makes explicit constraints that are implicit in both the protocol knowledge and the behaviors. It is shown that the approach facilitates the derivation of new operational constraints on protocol behavior. The new constraints lead to new types of protocol behavior, thereby yielding potentially valuable new conformance test cases. An application of the method to the Alternating Bit Protocol (ABP) (a canonical example in protocols research literature) is described.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Protocol conformance test generation using multiple UIO sequences with overlapping

This paper describes an optimization method for reducing the length of protocol conformance test sequences by overlapping test subsequences obtained using UIO sequences. It is shown that test sequences generated by this method are substantially shorter than those generated by other methods employing UIO sequences.

Formal methods for generating protocol conformance test sequences

The four major methods of conformance test generation reported in the literature are reviewed: transition tours; distinguishing sequences; characterizing sequences; and unique input/output sequences. These methods are used to test the control portion of a protocol specification. The conformance testing concepts developed in the standards world are summarized. Their relationship with the four formal methods is discussed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Probabilistic testing of protocols

Test sequences are used for the conformance testing of communication protocols to standards. This paper discusses a new approach to generating test sequences. The approach is based on probabilistic concepts about protocol state transitions and communication channels. The novel feature of the test sequences generated by this technique is that the most probable states of a protocol will be tested more promptly.

MAP 2.1 conformance testing tools

The major components of the MAP 2.1 conformance test system are described. Protocol conformance testing and program testing are compared. The scope and process of dynamic conformance testing is reviewed. The architecture of the tests system is presented, first in terms of the developing ISO (International Organization for Standardization) framework, and then in terms of run-time components. Several specific tools which comprise the test system are described. These tools include test engines, a high-level control tool, monitoring and analysis tools, and document handling tools. Benefits and limitations of the test tools are examined. Conclusions and suggestions for future efforts are provided. >

Protocol testing techniques

This paper deals with testing techniques for communication protocols. Several protocol testing support systems designed for testing protocol layers of the ISO/OSI (International Standards Organisation/Open Systems Interconnection) Reference Model are introduced. These systems are mostly designed for the conformance testing of protocols produced by third parties, which is intended not only to provide test services for both suppliers and users but also to meet the requirements of standards. Several test case generation methodologies are described, and the authors' experience in using one of the methods is presented.

Formal specification-based conformance testing

The paper discusses the use of formal specifications for conformance testing of OSI protocols and divides the discussion in two parts: test design and tester design. A draft standard formal specification of the Class 4 transport protocol in Estelle is taken as the starting point for test design. The test design technique used to derive a conformance test suite is semi-formal based in part on the formal specification and also the informal specification. The tests obtained are expressed in natural language. In the tester design part, we discuss the distributed test architecture of ISO and give the detailed designs of upper and lower testers. It is shown that, in testing Class 4 transport implementations, a parametrized protocol implementation approach in the lower tester design, renders the tests easier to implement.