Rust-twins: Automatic Rust Compiler Testing through Program Mutation and Dual Macros Generation

This paper, based on the EDA software platform and VHDL description method, introduces how to use the FPGA chip to control the multiphase stepper motor. This design, through automatic logic compilation optimization and simulation test and compile, finally completed the function of the stepping motor subdivision drive controller. By testing, it can achieve controlling positive &negative and subdivision for stepper angle.

Compilers play an important role in software development, and it is quite necessary to perform abundant testing to ensure the correctness of compilers. A critical task in compiler testing is to validate the semantic-soundness property which requires consistence between semantics of source programs and behavior of target executables. For validating this property, one main challenging issue is generation of a test oracle. Most existing approaches fall into two main categories when dealing with this issue: reference-based approaches and assertion-based approaches. All these approaches have their weakness when new programming languages are involved or test automation is required. To overcome the weakness in the existing approaches, we propose a new automatic approach for testing compiler. Our approach is based on the technique of metamorphic testing, which validates software systems via so-called "metamorphic relations". We select the equivalence-preservation relation as the metamorphic relation and propose an automatic metamorphic testing framework for compiler. We also propose three different techniques for automatically generating equivalent source programs as test inputs. Based on our approach, we developed a tool called Mettoc. Our mutation experiments show that Mettoc is effective to reveal compilers' errors in terms of the semantic-soundness property. Moreover, the empirical results also reveal that simple approaches for constructing test inputs are not weaker than complicated ones in terms of fault-detection capability. We also applied Mettoc in testing a number of open source compilers, and two real errors in GCC-4.4.3 and UCC-1.6 respectively have been detected by Mettoc.

Introduction: The standard way to check the quality of a compiler is manual testing. However, it does not allow to cover a vast diversity of programs that can be written in a target programming language. Today, in addition to manual written tests there are many automated compiler testing methods, among which fuzzing is one of the most powerful and useful. A compiler fuzzer is a tool that generates a random program in a target language and checks how the compiler works in this language. Purpose: To develop a platform for compiler fuzzing and, based on it, to develop a tool for Kotlin compiler testing. Results: We have developed Backend Bug Finder which is a platform for compiler fuzzing is. We have chosen a mutation-based approach as a method for generating random programs. First, an existing program is entered to the mutator as the input to be then transformed in some way. Mutations can be both trivial, for example, replacing arithmetic operators with others, and complex, changing the structure of the program. Next, the resulting program is fed to the input of the compiler with the following check of its operation. The developed test oracle can detect three types of errors: crashes, miscompilations, and performance degradations. If an error is detected, the test case is fed into the post-processing module, where reduction and deduplication algorithms are applied. We have developed a tool for fuzzing the Kotlin language compiler based on the platform for its approbation, which showed the applicability of the proposed approach for finding errors in modern compilers. Practical relevance: Over a year and a half of work, our tool has found thousands of different Kotlin compiler bugs, more than 200 of which were sent to the developers, and more than 80 have been fixed.

This paper describes RandIR, a tool for differential testing of compilers using random instances of a given intermediate representation (IR). RandIR assumes no fixed target language but instead supports extensible IR-definitions through an internal IR-independent representation of operations. This makes it particularly well suited to test embedded compilers for multi-stage programming, which is our main use case. The ideas underlying our work, however, are more generally applicable. RandIR is able to automatically simplify failing instances of a test, a technique commonly referred to as shrinking. This enables testing with large random IR samples, thus increasing the odds of detecting a buggy behavior, while still being able to simplify failing instances to human-readable code.

The ever-increasing density of ASICs, the whole-sale switch to surface-mount technology, and the growing interest in multi-chip modules (MCM), have resulted in testable designs becoming a greater priority. Thus far, designers have considered testability as an issue which comes into play at the very end of the design cycle. However, in the ASIC design flow based on synthesis, it is essential that designers develop a test strategy and address testability issues concurrently with other activities in the design cycle. In this chapter, Test Synthesis and Automatic Test Pattern Generation (ATPG) using the Synopsys Test Compiler (TC) are discussed.

We describe our experience over the last 18 months on a compiler testing technology transfer project: taking the GraphicsFuzz research project on randomized metamorphic testing of graphics shader compilers, and building the necessary tooling around it to provide a highly automated process for improving the Khronos Vulkan Conformance Test Suite (CTS) with test cases that expose fuzzer-found compiler bugs, or that plug gaps in test coverage. We present this tooling for test automation - gfauto - in detail, as well as our use of differential coverage and test case reduction as a method for automatically synthesizing tests that fill coverage gaps. We explain the value that GraphicsFuzz has provided in automatically testing the ecosystem of tools for transforming, optimizing and validating Vulkan shaders, and the challenges faced when testing a tool ecosystem rather than a single tool. We discuss practical issues associated with putting automated metamorphic testing into production, related to test case validity, bug de-duplication and floating-point precision, and provide illustrative examples of bugs found during our work.

The accuracy of parsing execution directly affects the accuracy of semantic analysis, optimization and object code generation. Therefore, parser testing represents the basis of compiler testing. It should include tests for correct and expected, but also for unexpected and invalid cases. Techniques for testing the parser, as well as algorithms and tools for test sentences generation, are discussed in this paper. The methodology for initial testing of a newly developed compiler is proposed. Generation of negative test sentences by modifying the original language grammar is described. Positive and negative test cases generated by Grow, Purdom's algorithm with and without length control, CDRC-P algorithm and CDRC-P algorithm with length control are applied to the testing of L-IRL robot programming language. For this purpose two different tools for generation of test sentences are used. Based on the presented analysis of possible solutions, the appropriate method can be chosen for testing the parser for smaller grammars with many recursive rules.

When debugging programs, we often assume the debugger itself is correct. However, when it is not, it becomes hard to find bugs or lets developers search for bugs that are not even present. We thus propose a new approach to automatic testing of debuggers, inspired by differential testing of compilers. Our approach generates debugger actions to exercise the debugger and records a trace during the debugging session. By comparing traces of different debuggers against each other, we find deviating behavior and bugs. We evaluate our approach on the JavaScript debuggers of Firefox and Chromium and find 16 previously unreported bugs, four of which are already confirmed and fixed.

Symbolic execution has attracted significant attention in recent years, with applications in software testing, security, networking and more. Symbolic execution tools, like CREST, KLEE, FuzzBALL, and Symbolic PathFinder, have enabled researchers and practitioners to experiment with new ideas, scale the technique to larger applications and apply it to new application domains. Therefore, the correctness of these tools is of critical importance. In this paper, we present our experience extending compiler testing techniques to find errors in both the concrete and symbolic execution components of symbolic execution engines. The approach used relies on a novel way to create program versions, in three different testing modes-concrete, single-path and multi-path-each exercising different features of symbolic execution engines. When combined with existing program generation techniques and appropriate oracles, this approach enables differential testing within a single symbolic execution engine. We have applied our approach to the KLEE, CREST and FuzzBALL symbolic execution engines, where it has discovered 20 different bugs exposing a variety of important errors having to do with the handling of structures, division, modulo, casting, vector instructions and more, as well as issues related to constraint solving, compiler optimisations and test input replay.