Fault-oriented Test Generation Research Articles

Automated Test Case Generation for Digital System Designs: A Mapping Study on VHDL, Verilog, and SystemVerilog Description Languages

Researchers have proposed different methods for testing digital systems and circuits in the last couple of decades. The need for testing digital logic circuits has become more important than ever due to the growing complexity of such systems. During the design phase, testing is focusing on design defects, as well as manufacturing and wear out type of defects. Failures in digital systems could be caused, for example, by design errors, the use of inherently probabilistic devices, and manufacturing variability. As a way to test digital systems in a more efficient way, automated test generation has been proposed to automatically create tests that can quickly and accurately identify faulty components. Examples of such techniques are the sequential test generation, the scan path testing, and the random test generation techniques. With the research domain becoming more mature and growing, it is essential to systematically identify, analyze, and classify these contributions. We performed a systematic mapping study of automated test generation for digital circuits aimed at providing an overview of the application of these techniques. We focused on three of the most widely-used and well-supported hardware description languages (HDLs) for digital systems: Verilog, SystemVerilog, and VHDL. Our results suggest that the majority of the test generation methods for digital circuits are focused on the behavioral and register-transfer design levels. Fault-independent and fault-oriented test generation are the most frequently reported types of test generation methods, while HDL model simulation is the most common test generation technology used to search for test cases in these academic studies. While the results are suggesting a growing interest in this area, the majority of articles are published as conferences papers. Our results show that only 31% of the methods are implemented as software tools and only 63% of all contributions are actually generating executable test cases. This study makes three important contributions, (i) a state-of-the-art of test generation for digital system designs research is provided, (ii) the reported characteristics are identified in both the primary papers and experimental reports, (iii) gaps and opportunities for future test generation for digital system designs research are identified.

Test-Quality/Cost Optimization Using Output-Deviation-Based Reordering of Test Patterns

At-speed functional testing, delay testing, and n-detection test sets are being used today to detect deep submicrometer defects. However, the resulting test data volumes are too high; the 2005 International Roadmap for Semiconductors predicts that test-application times will be 30 times larger in 2010 than they are today. In addition, many new types of defects cannot be accurately modeled using existing fault models. Therefore, there is a need to model the quality of test patterns such that they can be quickly assessed for defect screening. Test selection is required to choose the most effective pattern sequences from large test sets. Current industry practice for test selection is based on fault grading, which is computationally expensive and must also be repeated for every fault model. Moreover, although efficient methods exist today, for fault-oriented test generation, there is a lack of understanding on how best to combine the test sets thus obtained, i.e., derive the most effective union of the individual test sets without simply taking all the patterns for each fault model. This paper presents the use of the output deviation as a surrogate coverage-metric for pattern modeling and test grading. A flexible, but general, probabilistic-fault model is used to generate a probability map for the circuit, which can subsequently be used for test-pattern reordering. The output deviations resulting from the probability map(s) are used as a coverage-metric to model test patterns; the higher the deviation, the better the quality of the test pattern. We show that, for the ISCAS benchmark circuits and as compared to other reordering methods, the proposed method provides ldquosteeperrdquo coverage curves for different fault models.

FOTG: fault-oriented stress testing of IP multicast

Network simulators provide a useful tool, for protocol evaluation. However, the results depend heavily on the simulated scenarios, especially for complex protocols such as multicast. There has been little work on scenario generation. In this work we present a fault-oriented test generation (FOTG) algorithm for automated stress testing of multicast protocols. FOTG processes an extended FSM model and uses a mix of forward and backward search techniques. Unlike traditional verification approaches, instead of starting from initial states, FOTG starts from a fault and uses cause-effect relations for automatic topology synthesis then uses backward implication to generate tests. Using FOTG we test various mechanisms commonly employed by multicast routing and validate our results through simulation.

The STRESS Method for Boundary-Point Performance Analysis of End-to-End Multicast Timer-Suppression Mechanisms

Evaluation of Internet protocols usually uses random scenarios or scenarios based on designers' intuition. Such approach may be useful for average-case analysis but does not cover boundary-point (worst or best-case) scenarios. To synthesize boundary-point scenarios a more systematic approach is needed.In this paper, we present a method for automatic synthesis of worst and best case scenarios for protocol boundary-point evaluation. Our method uses a fault-oriented test generation (FOTG) algorithm for searching the protocol and system state space to synthesize these scenarios. The algorithm is based on a global finite state machine (FSM) model. We extend the algorithm with timing semantics to handle end-to-end delays and address performance criteria. We introduce the notion of a virtual LAN to represent delays of the underlying multicast distribution tree. The algorithms used in our method utilize implicit backward search using branch and bound techniques and start from given target events. This aims to reduce the search complexity drastically. As a case study, we use our method to evaluate variants of the timer suppression mechanism, used in various multicast protocols, with respect to two performance criteria: overhead of response messages and response time. Simulation results for reliable multicast protocols show that our method provides a scalable way for synthesizing worst-case scenarios automatically. Results obtained using stress scenarios differ dramatically from those obtained through average-case analyses. We hope for our method to serve as a model for applying systematic scenario generation to other multicast protocols.

A diagnostic test generation procedure based on test elimination by vector omission for synchronous sequential circuits

We propose a procedure for generating test sequences for diagnosis of synchronous sequential circuits based on stuck at faults. In this procedure, we avoid the conventional fault-oriented test generation process by observing that a sequence to distinguish two faults can be obtained from a sequence T that detects both of the faults (such as a test sequence for fault detection) by changing T so as to "undetect" one of the faults, or change the time units or outputs where the fault is detected. To achieve this goal, the proposed procedure eliminates parts of T so as to render some of the faults undetected, or change their detection times or outputs. In the case where faults become undetected by the modified sequence, the detected faults are distinguished from the faults left undetected by the modified sequence based on pass/fall information. A pass/fall dictionary based on modified test sequences is proposed for this case. Alternatively, a standard dictionary can be used, and the proposed procedure can be used to change the time units or outputs where faults are detected in order to distinguish them. We present experimental results to demonstrate the levels of resolution that can be obtained by the proposed procedure with the proposed pass/fail dictionary, and the number of sequences required for this purpose.

Functional versus random test generation for sequential circuits

This article presents a test generation method for sequential circuits based on their synthesis specifications as finite state machines (FSM) and provides comparison with random test generation. The finite state machines are represented by their state transition graph (STG). The test generation method is performed in two phases. The first phase is functional. It generates a test sequence which is one of the shortest input sequences going through all the transitions of the state transition graph machine. This sequence provides a high fault coverage of stuck-at faults on the synthesized circuit compared to a randomly generated test sequence. This fault coverage is very close to the ones of other sequences derived by fault-oriented test generation approaches [9], [10], although these latter sequences are much longer.