A Control-Oriented Coverage Metric and its Evaluation for Hardware Designs

Verma Verma

doi:10.3844/jcssp.2009.302.310

Abstract

Problem statement: Dynamic verification, the use of simulation to determine design correctness, is widely used due to its tractability for large hardware designs. A serious limitation of dynamic techniques is the difficulty in determining whether or not a test sequence is sufficient to detect all likely design errors. Coverage metrics are used to address this problem by providing a set of goals to be achieved during the simulation process; if all coverage goals are satisfied then the test sequence is assumed to be complete. Coverage metrics hence evaluate the ability of a test sequence to detect design errors and are essential to the verification process. A key source of difficulty in determining error detection is that the control-flow path traversed in the presence of an error cannot be determined. This problem becomes particularly difficult in case of typical industrial designs involving interaction of control flow paths between concurrent processes. Error detection can only be accurately determined by exploring the set of all control-flow paths, which may be traversed as a result of an error. Also, there is no technique to identify a correlation between coverage metrics and hardware design quality. Approach: We present a coverage metric that determined the propagation of error effects along all possible erroneous control-flow paths across processes. The complexity of exploring multiple control-flow paths was greatly alleviated by heuristically pruning infeasible control-flow paths using the algorithm that we present. We also presented a technique to evaluate coverage metric by examining its ability to ensure the detection of real design errors. We injected errors in the design to correlate their detection with the coverage computed by our metric. Results: Our coverage metric although analyzed all control-flow paths it pruned the infeasible ones and eliminated them from coverage consideration, hence reducing the complexity of generating tests meant to execute them. The metric also correlated better with detection of design errors than some well-studied metrics do. Conclusion: The proposed coverage metric provided high accuracy in measurement of coverage in designs that contain complex control-flow with concurrent processes. It is superior at detecting design error when compared with the metrics it was compared with.

Highlights

Verification is known to be an expensive and timeconsuming part of the design process
A general framework of the dynamic verification process is shown in Fig. 1 where the process starts on the left side with an executable design description in a Hardware Description Language (HDL)
We present a technique to estimate the correlation between coverage metrics and design error detection

Summary

Introduction

Verification is known to be an expensive and timeconsuming part of the design process. The subject of this study, involves the use of simulation to verify design correctness. The three main steps of the process are test generation, simulation and response evaluation. Simulation is performed using a software tool but test generation and response evaluation, are often heavily dependent on manual interaction making them costly. The most significant weakness of dynamic verification techniques is the difficulty of determining the error detection ability of a test sequence. We refer to a test sequence as being complete if it detects all potential design errors. The effectiveness of dynamic verification depends on the ability to determine whether or not a test sequence is complete. An incomplete test sequence will lead to the possibility of design errors in the final product and reduced system quality

Methods

Results

Conclusion