A Comparison of Fault Detection Efficiency Between Adaptive Random Testing and Greedy Combinatorial Testing for Control Logics in Nuclear Industrial Distributed Control Systems

Cheng Zong,Shiyong Shuang,Ziyuan Wang,Yanliang Zhang,Yongming Yao

doi:10.1109/access.2021.3087165

Abstract

Due to the complexity of the nuclear industrial distributed control system (DCS), input-domain testing techniques, including random testing and combinatorial testing, are usually utilized to test the control logics in nuclear industrial DCS. To improve the fault detection efficiency of random testing, the adaptive random testing technique selects a test case that significantly differs from all existing test cases. Similarly, to improve the fault detection efficiency of combinatorial testing, the greedy combinatorial testing technique adopts a greedy strategy to generate test cases that cover more uncovered tuple-combinations of parametric values. In this paper, we designed an experiment to compare the fault detection efficiency between adaptive random testing technique and greedy combinatorial testing technique for control logics of nuclear industrial DCS. Through the analysis of the fault detection ratios, the f-measure values, and the values of average percent of faults detected (APFD) on two experimental subjects, including the commonly used benchmarks in the field of Boolean-specification testing as well as a group of Boolean expressions extracted from the control logics in nuclear industrial DCS, the experimental results give us the following conclusions: (1) If the test suites' sizes are relatively small, the fault detection efficiencies of the two techniques are very close though there is a slight advantage in adaptive random testing; (2) With the gradual increase of test suites' sizes, the fault detection efficiency of greedy combinatorial testing is beyond adaptive random testing gradually. Such a result can help us select the appropriate testing techniques in the testing of the control logics in nuclear industry DCS.

Highlights

The nuclear industrial distributed control system (DCS) is applied in the field of safety, which takes the initiative and protective action to protect the critical core parts of nuclear power units from damage when some crucial parameters of nuclear power units exceed the limitation, and reduce the probability of nuclear accidents
Under the condition of unlimited testing resources, we can answer the second question by comparing f-measure values and average percent of faults detected (APFD) values of adaptive random testing and greedy combinatorial testing
The adaptive random testing technique has certain similarities with the greedy combinatorial testing technique; the former requires the maximization of the Hamming distance between different test cases, while the latter requires the maximization of the difference of the tuplecombinations covered between different test cases

Summary

Introduction

The nuclear industrial distributed control system (DCS) is applied in the field of safety, which takes the initiative and protective action to protect the critical core parts of nuclear power units from damage when some crucial parameters of nuclear power units exceed the limitation (such as pressure or pressure difference, temperature or temperature difference, liquid level, etc.), and reduce the probability of nuclear accidents. The fault diagnosis results for the nuclear industrial distributed control system (DCS) show that the root causes of a large number of faults are incorrect design or incorrect implementation of control logics in this type of system [1]. To ensure the correctness of the control logics in the nuclear industrial DCS and to. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. The input-domain testing techniques are usually used to test the control logics in nuclear industrial DCS

Methods

Results

Conclusion