Generate High-Coverage Unit Test Data Using the LLM Tool

Abstract

Listen

Translate article

Unit testing is a critical phase in the software development lifecycle, essential for ensuring the quality and reliability of code. However, the manual creation of unit test scripts and the preparation of corresponding test data can be a time consuming and labor intensive process. To address these challenges, several automated approaches have been explored, including search based, constraint based, random-based, and symbolic execution based techniques for generating unit tests. In recent years, the rapid advancement of large language models (LLMs) has opened new avenues for automating various tasks, including the automatic generation of unit test scripts and test data. Despite their potential, using LLMs in a straightforward manner to generate unit tests may lead to low test coverage. This means that a significant portion of the source code, including specific statements or branches, may remain untested, which can reduce the effectiveness of the tests. To overcome this limitation, the paper presents a novel approach that not only automates the generation of unit test scripts and test data but also improves test coverage. The proposed solution begins by using an LLM tool (such as ChatGPT) to generate initial unit test scripts and data from the source code. To enhance test coverage, the specification document of the source code is also input into the LLM to generate additional test data. Following this, a coverage checking tool is used to evaluate the test coverage and identify untested statements or branches. The LLM is then applied again to generate new test data aimed specifically at addressing these gaps. The initial experimental results indicate that this method significantly improves test coverage, demonstrating its potential to enhance automated unit testing processes.