Automated Testing of a TCG Frontend for Qemu

Abstract

Implementing new target architecture in Qemu involves creating a frontend of the TCG dynamic binary translator to this architecture. Testing is necessary to verify the correctness of that translator component. Currently, existing TCG frontend testing systems use an approach based on comparison with an oracle. This oracle has the same processor architecture and can be a real processor, high-fidelity emulator, or another binary translator. Unfortunately, these oracles are not always available. This paper is devoted to testing a target architecture implementation in Qemu when the necessary oracle is not available. The proposed approach is based on the fact that the binutils package and C compiler are usually available even for rarely used processor architectures. A program written in a high-level programming language is expected to run in the same manner on different processor architectures if it is possible to avoid its undefined or implementation-defined behavior. This allows two different executables to be compared on a developer’s machine and a virtual machine being tested. The units to be compared are entities of the high-level programming language used to write tests; in our case, tests are written in C. The approach is implemented in CPU Testing Tool (c2t) and is part of the Qemu Development Toolkit (QDT); its source code is available at https://github.com/ispras/qdt . c2t is implemented in Python and supports Qemu testing in both full-system and user-level emulation modes. This tool is suitable for testing TCG frontends generated by automated TCG frontend generation systems or implemented manually.