Random testing of C compilers based on test program generation by equivalence transformation

Abstract

This paper presents a new method of generating test programs for random testing of C compilers based on equivalence transformation on C programs. Although equivalence transformation on programs is a promising way of generating new test programs without undefined behavior, existing methods needed valid test programs used as seeds, and they applied only addition/deletion of statements to/from unreachable portion of the seed test programs. On the other hand, our method generates a vast variety of test programs starting from a trivial initial program by repeatedly applying equivalence transformation, which works on live codes. Furthermore, our method can control the resulting operand values of the subexpressions in test programs, which contributes to a higher bug detection ability. A random test systems, Orange4, based on our method has detected bugs in the latest development versions of GCC-6.0.0 and LLVM-3.9.