RNNtcs: A test case selection method for Recurrent Neural Networks

Abstract

Recurrent Neural Network (RNN) is a typical feedback neural network, which is particularly effective in processing time-series data tasks such as image description, text generation and classification, etc. However, its safety and quality issues have aroused strong concern in academia and industry. One of the most commonly used methods to ensure RNN-driven software quality is testing and optimizing RNN. Its core idea is to introduce human efforts to label test cases so that many labeled test cases can be used to trigger faults in RNNs. But this brings a huge overhead. Therefore, it is of great importance to study the test case selection method for RNN. Current research on test case selection for RNN has achieved some results from the stateful view of RNN, but there are still some problems, such as a low bug detection rate, poor inclusiveness, and lack of diversity.In this paper, according to the structural characteristics of RNN, we propose a method of RNN test case selection, RNNtcs, which combines clustering and uncertainty, in order to select test cases that can touch RNN bugs as much as possible, thus reducing the cost of labeling. In order to evaluate RNNtcs, we conduct experiments on popular image datasets, text datasets and RNN models. The experimental results show that RNNtcs is superior to the state-of-the-art approach DeepState in bug detection effectiveness, bug detection inclusiveness and bug detection diversity. The test cases selected by RNNtcs can effectively improve the robustness of the RNN model.