SRTEF: Test Function Recommendation With Scenarios and Latent Semantic for Implementing Stepwise Test Case

Abstract

Implementing test cases as programs to automate test execution is a popular testing practice. Current industrial practices usually use test functions to implement the test steps of a test case and then to compose the executable test case by choosing the test functions to call manually. It is time-consuming and could lead to invalid test results by selecting inappropriate test functions. In this article, we propose an automatic test function recommendation approach named Scenario-based Recommendation of TEst Function (SRTEF). Given a test step of a test case, SRTEF uses the weighted description similarity and the scenario similarity to recommend test functions. The description similarity utilizes the deep structured semantic model (DSSM) to measure the relatedness between a test step and a test function by their literal descriptions. The test scenario and the test function usage scenario are considered to calculate the scenario similarity. SRTEF has been successfully applied in Huawei. The systematic experiments have been conducted to evaluate SRTEF by using the dataset from Huawei and comparing with BiInformation source-based KnowledgE Recommendation (BIKER), reported as the best approach so far. The results show that SRTEF outperforms BIKER with significant positive ratios consistently in all the three selection strategies, i.e., Top-3, Top-5, and Top-10. The DSSM shows its advantage over word embedding by the double performance of capturing the semantic relatedness in SRTEF.