Test Input Generation Research Articles

Test input generation with java PathFinder

We show how model checking and symbolic execution can be used to generate test inputs to achieve structural coverage of code that manipulates complex data structures. We focus on obtaining branch-coverage during unit testing of some of the core methods of the red-black tree implementation in the Java TreeMap library, using the Java PathFinder model checker. Three different test generation techniques will be introduced and compared, namely, straight model checking of the code, model checking used in a black-box fashion to generate all inputs up to a fixed size, and lastly, model checking used during white-box test input generation. The main contribution of this work is to show how efficient white-box test input generation can be done for code manipulating complex data, taking into account complex method preconditions.

Generating test inputs for embedded control systems

Embedded control systems are growing rapidly, and there is a need for short design cycles. As a result, there is increasing interest in effective methods for automatic test generation. The authors present a new method for leveraging existing simulation models, involving genetic algorithms, for embedded control system designs to generate test inputs automatically, thereby eliminating the time-consuming task of creating them manually.

Generating test data from state‐based specifications

AbstractAlthough the majority of software testing in industry is conducted at the system level, most formal research has focused on the unit level. As a result, most system‐level testing techniques are only described informally. This paper presents formal testing criteria for system level testing that are based on formal specifications of the software. Software testing can only be formalized and quantified when a solid basis for test generation can be defined. Formal specifications represent a significant opportunity for testing because they precisely describe what functions the software is supposed to provide in a form that can be automatically manipulated.This paper presents general criteria for generating test inputs from state‐based specifications. The criteria include techniques for generating tests at several levels of abstraction for specifications (transition predicates, transitions, pairs of transitions and sequences of transitions). These techniques provide coverage criteria that are based on the specifications and are made up of several parts, including test prefixes that contain inputs necessary to put the software into the appropriate state for the test values. The test generation process includes several steps for transforming specifications to tests. These criteria have been applied to a case study to compare their ability to detect seeded faults. Copyright © 2003 John Wiley & Sons, Ltd.

Declarative paradigm of test coverage

Two facts about declarative programming prevent the application of conventional testing methods. First, the classical test coverage measures such as statement, branch or path coverage, cannot be used, since in declarative programs no control flow notion exists. Second, there is no widely accepted language available for formal specification, since predicate logic, which is the most common formalism for declarative programming, is already a very high-level abstract language. This paper presents a new approach exending previous work by the authors on test input generation for declarative programs. For this purpose, the existing program instrumentation notion is extended and a new logic coverage measure is introduced. The approach is mathematically formalized and the goal of achieving 100% program logic coverage controls the automatic test input generation. The method is depicted by means of logic programming; the results are, however, generally applicable. Finally, the concepts introduced have been used practically within a test environment. © 1998 John Wiley & Sons, Ltd.

Using modern software tools to design, simulate and test a Level 1 trigger sub-system for the D Zero detector

The work presents a system which uses a commercial spreadsheet program and commercial hardware on an IBM PC to develop and test a track finding system for the D Zero Level 1 scintillating fiber trigger. The trigger system resides in a VME crate. This system allows the user to generate test input, write the pattern to the hardware, simulate the results in software, read the hardware result, compare the results and inform the user of any differences.

Simultaneous signature and syndrome compression

A commonly used organization for built-in self-test of VLSI (very large-scale integration) circuits uses complete or pseudorandom test input generators followed by output data reduction. Two compression techniques which have been used are polynomial division (signature) and ones counting (syndrome). The simultaneous use of both of these approaches in parallel is investigated. Analytic and enumerative results indicate that the number of error patterns which are missed by both methods together is nearly the theoretical minimum. The conclusion extends to signature compression combined with any other counter-based compression method such as the use of Walsh spectral coefficients. Some suggestions for CAD (computer-aided design) implementations of test design are given based on these results. >

シーケンス制御系の設計修正に伴う機能変化の検証

The design of industrial sequence control systems, such as power plants, is often carried out by the modification of an old design. The paper describes a new method to verify whether the functional change caused by the modification may influence the unmodified portion or not.Assume that a portion A of an original system is modified to A', and the remaining unmodified portion is B. The portion A and the modified portion A' include a controller part and controlled devices.Both A and A' are converted into timed petri nets: TPN (A) and TRN (A'). TPN (A) and TPN (A') have inputs I which are signals from B and outputs O which are signals to B. As B is affected by the outputs O, the change of outputs O is examined.In order to check the difference between A and A', TPN (A) and TPN (A') are converted into automata with the reachability tree. In this conversion, only stable markings, in which no transition is ‘firable’ if the inputs (I) never change, are selected as the states of the automata. The inputs, the state transitions and the outputs of the automata are obtained from the inputs (I), the change of markings and the outputs (O) of TPN (A) and TPN (A'), respectively.The paper shows that the functional change is easy to verify by comparing the automata's output functions of the old design and the new one. Besides, the automata are used to generate test inputs to check the functional change.

On the Automated Generation of Program Test Data

Software validation through testing will continue to be a very important tool for ensuring correctness of large scale software systems. Automation of testing tools can greatly enhance their power and reduce testing cost. In this paper, techniques for automated test data generation are discussed. Given a program graph, a set of paths are identified to satisfy some given testing criteria. When a path or a program segment is specified, symbolic execution is used for generating input constraints which define a set of inputs for executing this path or segment. Problems encountered in symbolic execution are discussed. A new approach for resolving array reference ambiguities and a procedure for generating test inputs satisfying input constraints are proposed. References to arrays are recorded in a table. during symbolic execution and ambiguities are resolved when test data are generated to evaluate the subscript expressions. The implementation of a test data generator for Fortran programs incorporating these techniques is also described.