Finding Compiler Bugs Duplicates by Generating Witness Programs

Abstract

Programming language compilers are complex software projects, the quality of which directly affects the quality of programs created by them. Therefore, compilers are subject to increased requirements for such software quality characteristics as functional suitability, reliability, performance level, safety, etc. To ensure quality, various methods are used: user and manual testing, tools for automatic error detection. And due to many methods of searching for bugs often situations arise when the same error, especially if its cause is trivial, is detected repeatedly, and the test cases that lead to it can be completely different from each other. Such test cases are called duplicates, and their determination is an urgent and acute problem, since their manual search requires a large amount of human resources. Algorithms for automatically finding duplicates would greatly simplify the process of developing and maintaining a compiler. The main idea of the approach for finding duplicates presented in the article is that the causes of the same errors are located in the same place in the compiler source code. To search for this place, the method of generating witness programs is used: for each test program containing an error, similar programs are generated that do not contain it. After that, metrics are calculated based on the source code coverage of the compiler and a list of source code files that are potentially causing the compiler to fail is formed. If these lists for two test programs are similar in terms of the proximity metric, then the test programs are considered duplicates. The proposed approach was developed and implemented for the compiler of the Kotlin programming language. Testing has shown the applicability of the proposed approach for solving the problem of finding duplicate errors of compilers of programming languages