Weighted Reward for Reinforcement Learning based Test Case Prioritization in Continuous Integration Testing

Abstract

Test Case Prioritization (TCP) based on the continuous decision of Reinforcement Learning (RL) has achieved a successful application for test cases optimization in Continuous Integration (CI). The reward functions of RL describe how a test case ought to be executed in next integration, of which the design is usually based on the historical executions of the test case. The Average Percentage of Historical Failure (APHF) had been considered as one of the best reward function which has a strong correlation with the recent failure executions of a test case. However, for a test case with many historical failures but passes in recent cycles, the APHF value may be low. In this paper, two novel reward functions are proposed focusing on the impact of failure position in test case history execution sequence, which are the Average Position Exponential Weight (APEW) reward function and the Average Position Quadratic Weight (APQW) reward function, respectively. Both APEW and APQW carry out weight design of failure position but with different weights. We theoretically prove the issue of the only strong correlation with recent failure executions, and also prove that both proposed reward functions can reflect the quantity of historical failures and the distribution of these failures. Experimental verification on 10 industrial-level data sets show that the proposed reward functions can effectively improve the fault detection capability of test cases.