A lightweight adaptive random testing method for deep learning systems

Abstract

AbstractIn recent years, deep learning (DL) systems are increasingly used in the safety‐critical fields such as autonomous driving, medical diagnosis, and financial service. Although these systems have demonstrated an outstanding performance in enhancing the accuracy of decision‐making, they pose significant challenges to the trustworthiness due to their limited interpretability and inherent uncertainty. Adaptive random testing (ART) has been proved as an effective approach for ensuring the reliability of DL systems. However, existing ART methods for DL systems incur a heavy overhead in test case selection due to the computation of distances. To address this issue, we propose a lightweight adaptive random testing (Lw‐ARTDL) method for DL systems. In our improved algorithm, we employ the K‐Means technique to divide the entire test suite into several subsets. Then, for a candidate test case, we only calculate distances between it and the test cases within the category to which it belongs. This partition strategy ensures that the selected test cases are more representative while significantly reducing the computational cost. To validate the proposed algorithm, the comparison experiments between Lw‐ARTDL and the original ARTDL algorithm are conducted on two typical DL systems. The experimental results show that Lw‐ARTDL significantly reduces the overhead of failure detection, and exhibits stronger failure detection capability compared to ARTDL in most similarity metrics.