Bytecode Interpreter Research Articles

Python meets JIT compilers: A simple implementation and a comparative evaluation

AbstractDeveloping a just‐in‐time (JIT) compiler can be a daunting task, especially for a language as flexible as Python. While PyPy, powered with JIT compilation, can often outperform the official pure interpreter, CPython, by a noteworthy margin, its popularity remains far from comparable to that of CPython due to some issues. Given that an easier‐to‐deploy and better‐compatible JIT compiler would benefit more Python users, we have developed comPyler, a simple JIT compiler functioning as a CPython extension and intended to convert frequently interpreted CPython bytecode into equivalent machine code. Designed with good compatibility in mind, it does not alter CPython's internal data structures or external interfaces. Based on LLVM's mature infrastructure, it can be readily ported to almost all platforms. Compared with CPython, it achieved the highest speedup of 2.205, with an average of 1.093. Despite its relatively limited effect, comPyler incurs low development costs. As a baseline compiler, it also sheds light on the improvement attainable by optimizing solely the overhead of bytecode interpretation. Furthermore, as there is still a dearth of empirical research covering the multitude of JIT compilers available for Python, we have conducted a performance study that examines Jython, IronPython, PyPy, GraalPy, Pyston, Pyjion, and our comPyler. Our research takes into account not only the benchmark speed for various time windows but also the boot latency and memory footprint. Through this comprehensive study, our objective is to assist developers in gaining a better understanding of the effects of distinct JIT compilation techniques and to aid users in making informed decisions when choosing among different Python implementations.

Tidy Tuples and Flying Start: fast compilation and fast execution of relational queries in Umbra

Although compiling queries to efficient machine code has become a common approach for query execution, a number of newly created database system projects still refrain from using compilation. It is sometimes claimed that the intricacies of code generation make compilation-based engines too complex. Also, a major barrier for adoption, especially for interactive ad hoc queries, is long compilation time. In this paper, we examine all stages of compiling query execution engines and show how to reduce compilation overhead. We incorporate the lessons learned from a decade of generating code in HyPer into a design that manages complexity and yields high speed. First, we introduce a code generation framework that establishes abstractions to manage complexity, yet generates code in a single fast pass. Second, we present a program representation whose data structures are tuned to support fast code generation and compilation. Third, we introduce a new compiler backend that is optimized for minimal compile time, and simultaneously, yields superior execution performance to competing approaches, e.g., Volcano-style or bytecode interpretation. We implemented these optimizations in our database system Umbra to show that it is possible to unite fast compilation and fast execution. Indeed, Umbra achieves unprecedentedly low query latencies. On small data sets, it is even faster than interpreter engines like DuckDB and PostgreSQL. At the same time, on large data sets, its throughput is on par with the state-of-the-art compiling system HyPer.

Design, implementation, and application of GPU-based Java bytecode interpreters

We present the design and implementation of GVM, the first system for executing Java bytecode entirely on GPUs. GVM is ideal for applications that execute a large number of short-living tasks, which share a significant fraction of their codebase and have similar execution time. GVM uses novel algorithms, scheduling, and data layout techniques to adapt to the massively parallel programming and execution model of GPUs. We apply GVM to generate and execute tests for Java projects. First, we implement a sequence-based test generation on top of GVM and design novel algorithms to avoid redundant test sequences. Second, we use GVM to execute randomly generated test cases. We evaluate GVM by comparing it with two existing Java bytecode interpreters (Oracle JVM and Java Pathfinder), as well as with the Oracle JVM with just-in-time (JIT) compiler, which has been engineered and optimized for over twenty years. Our evaluation shows that sequence-based test generation on GVM outperforms both Java Pathfinder and Oracle JVM interpreter. Additionally, our results show that GVM performs as well as running our parallel sequence-based test generation algorithm using JVM with JIT with many CPU threads. Furthermore, our evaluation on several classes from open-source projects shows that executing randomly generated tests on GVM outperforms sequential execution on JVM interpreter and JVM with JIT.

Invalidating Analysis Knowledge for Code Virtualization Protection Through Partition Diversity

To protect programs from unauthorized analysis, virtualize the code based on Virtual Machine (VM) technologies is emerging as a feasible method for accomplishing code obfuscation. However, in some State-of-the-art VM-based protection approaches, the set of virtual instructions and bytecode interpreters are fixed across the whole programs. This means an experienced attacker could extract the mapping information between virtual instructions and native code from programs, and use this knowledge to uncover the mapping relationships in similar protecting applications. To address this problem, we present CoDiver (Code Virtualization Protection with Diversity), a novel VM-based code obfuscation system in this paper. The main idea of our approach is to obfuscate the mapping between the opcodes of bytecode instructions and their semantics. To achieve this goal, we first turn every protected code region into multiple parts by partition proceeding, randomize the mapping of opcodes and their semantics of each part. By this way, we could translate the bytecode instruction into different native code in different sections of the obfuscated code. This method could increase the diversity of program behavior significantly. As a result, it will be useless to learn the mapping relationship between bytecode and native code of some other programs, then migrate it into a new program. We build a prototype of CoDiver and tested it on a set of real-world applications. Experimental results show that as compared with two state-of-the-art VM-based code obfuscation approaches, our approach is more effective and could provide stronger protection with comparable runtime overhead and code size.

Making Compiling Query Engines Practical

Compiling queries to machine code is a very efficient way for executing queries. One often overlooked problem with compilation is the time it takes to generate machine code. Even with fast compilation frameworks like LLVM, generating machine code for complex queries often takes hundreds of milliseconds. Such durations can be a major disadvantage for workloads that execute many complex, but quick queries. To solve this problem, we propose an adaptive execution framework, which dynamically switches from interpretation to compilation. We also propose a fast bytecode interpreter for LLVM, which can execute queries without costly translation to machine code and dramatically reduces the query latency. Adaptive execution is fine-grained, and can execute code paths of the same query using different execution modes. Our evaluation shows that this approach achieves optimal performance in a wide variety of settings—low latency for small data sets and maximum throughput for large data sizes. Besides compilation time, we also focus on debugging, which is another important challenge of compilation-based query engines. To address this problem, we present a novel, database-specific debugger for compiling query engines.

Making extreme computations possible with virtual machines

State-of-the-art algorithms generate scattering amplitudes for high-energy physics at leading order for high-multiplicity processes as compiled code (in Fortran, C or C++). For complicated processes the size of these libraries can become tremendous (many GiB). We show that amplitudes can be translated to byte-code instructions, which even reduce the size by one order of magnitude. The byte-code is interpreted by a Virtual Machine with runtimes comparable to compiled code and a better scaling with additional legs. We study the properties of this algorithm, as an extension of the Optimizing Matrix Element Generator (O'Mega). The bytecode matrix elements are available as alternative input for the event generator WHIZARD. The bytecode interpreter can be implemented very compactly, which will help with a future implementation on massively parallel GPUs.

Abstract Interpretation of Java Bytecode for Immutability Analysis

Even though immutability is a desirable property, especially in a multi-threaded environment, implementing immutable Java classes is surprisingly hard because of a lack of language support. We present a static analysis tool using abstract bytecode interpretation that checks Java classes for compliance with a set of rules that together constitute state-based immutability. Being realized as a Find Bugs plug in, the tool can easily be integrated into most IDEs and hence the software development process. Our evaluation on a large, real world codebase shows that the average run- time effort for a single class is in the range of a few milliseconds, with only a very few statistical spikes.

Pycket: a tracing JIT for a functional language

We present Pycket, a high-performance tracing JIT compiler for Racket. Pycket supports a wide variety of the sophisticated features in Racket such as contracts, continuations, classes, structures, dynamic binding, and more. On average, over a standard suite of benchmarks, Pycket outperforms existing compilers, both Racket's JIT and other highly-optimizing Scheme compilers. Further, Pycket provides much better performance for Racket proxies than existing systems, dramatically reducing the overhead of contracts and gradual typing. We validate this claim with performance evaluation on multiple existing benchmark suites. The Pycket implementation is of independent interest as an application of the RPython meta-tracing framework (originally created for PyPy), which automatically generates tracing JIT compilers from interpreters. Prior work on meta-tracing focuses on bytecode interpreters, whereas Pycket is a high-level interpreter based on the CEK abstract machine and operates directly on abstract syntax trees. Pycket supports proper tail calls and first-class continuations. In the setting of a functional language, where recursion and higher-order functions are more prevalent than explicit loops, the most significant performance challenge for a tracing JIT is identifying which control flows constitute a loop---we discuss two strategies for identifying loops and measure their impact.

Hardware/software co-design for a high-performance Java Card interpreter in low-end embedded systems

Java Card is a Java running environment specific for smart cards. In such low-end embedded systems, the execution time of the applications is an issue of first order. One of the components of the Java Card Virtual Machine (JCVM) playing an important role in the execution speed is the bytecode interpreter. In Java systems the main technique for speeding-up the interpreter execution is the Just-In-Time compilation (JIT), but this resource consuming technique is inapplicable in systems with as restricted resources available as in smart cards.This paper presents a hardware/software co-design solution for the performance improvement of the interpreter. In the software domain, we adopted a pseudo-threaded code interpreter that allows a better run-time performance with a small amount of additional code. In the hardware domain, we proceeded moving parts of the interpreter into hardware, giving origin to a Java Card interpreter based on an application specific instruction set processor.

Surgical precision JIT compilers

Just-in-time (JIT) compilation of running programs provides more optimization opportunities than offline compilation. Modern JIT compilers, such as those in virtual machines like Oracle's HotSpot for Java or Google's V8 for JavaScript, rely on dynamic profiling as their key mechanism to guide optimizations. While these JIT compilers offer good average performance, their behavior is a black box and the achieved performance is highly unpredictable. In this paper, we propose to turn JIT compilation into a precision tool by adding two essential and generic metaprogramming facilities: First, allow programs to invoke JIT compilation explicitly. This enables controlled specialization of arbitrary code at run-time, in the style of partial evaluation. It also enables the JIT compiler to report warnings and errors to the program when it is unable to compile a code path in the demanded way. Second, allow the JIT compiler to call back into the program to perform compile-time computation. This lets the program itself define the translation strategy for certain constructs on the fly and gives rise to a powerful JIT macro facility that enables "smart" libraries to supply domain-specific compiler optimizations or safety checks. We present Lancet, a JIT compiler framework for Java bytecode that enables such a tight, two-way integration with the running program. Lancet itself was derived from a high-level Java bytecode interpreter: staging the interpreter using LMS (Lightweight Modular Staging) produced a simple bytecode compiler. Adding abstract interpretation turned the simple compiler into an optimizing compiler. This fact provides compelling evidence for the scalability of the staged-interpreter approach to compiler construction. In the case of Lancet, JIT macros also provide a natural interface to existing LMS-based toolchains such as the Delite parallelism and DSL framework, which can now serve as accelerator macros for arbitrary JVM bytecode.

TinyVM: an energy‐efficient execution infrastructure for sensor networks

SUMMARYEnergy‐efficient implementation techniques for virtual machines (VMs) have received little attention yet: conventional wisdom claims that VMs have a diametrical effect on energy consumption, and VM‐based applications are therefore short‐lived. In this paper, we argue that bytecode interpretation is affordable if we synthesize VMs specifically for energy efficiency. We present TinyVM, an execution infrastructure that seamlessly integrates with C and nesC/TinyOS‐based programming environments. TinyVM achieves high code density through the use of compressed bytecode as the primary program representation. Compressed bytecode allows rapid application deployment with low communication overhead. TinyVM executes compressed bytecode in place, which eliminates the need for a decompression stage and thereby reduces memory consumption on sensor nodes. Our infrastructure automates the creation of energy‐efficient application‐specific VMs. Applications are partitioned in machine code, bytecode, and VM instruction set extensions. Partitioning is manually controlled and/or fully guided by a discrete optimization problem that produces a partitioning with lowest energy consumption for a given program size limit. We provide experimental results for sensor network benchmarks and for selected applications on various CPU architectures including Atmega128‐based motes and the ARM‐based Intel iMote2. TinyVM has been released under the GNU General Public License. Copyright © 2011 John Wiley & Sons, Ltd.

Reconstruction of Type Information from Java Bytecode for Component Compatibility

The Java type system is strictly checked by both the compiler and the runtime bytecode interpreter of the JVM. These mechanisms together guarantee appropriate usage of class instances. Using modern component systems can however circumvent these static checks, because incompatible versions of classes can be bound together during component installation or update. Such problematic bindings result in ClassCastException or NoSuchMethodException runtime errors. In this paper we describe a representation of Java language types suitable for checking component compatibility. The presented approach applies various bytecode handling techniques to reconstruct a representation of the Java types contained in a component implementation, using different sources of class data. The representation is then used during build- and run-time type system verifications with the aim to prevent these kinds of errors. We have successfully applied this approach to prevent OSGi component incompatibilities.

ReCaml

To fix bugs or to enhance a software system without service disruption, one has to update it dynamically during execution. Most prior dynamic software updating techniques require that the code to be changed is not running at the time of the update. However, this restriction precludes any change to the outermost loops of servers, OS scheduling loops and recursive functions. Permitting a dynamic update to more generally manipulate the program's execution state, including the runtime stack, alleviates this restriction but increases the likelihood of type errors. In this paper we present ReCaml, a language for writing dynamic updates to running programs that views execution state as a delimited continuation. ReCaml includes a novel feature for introspecting continuations called match_cont which is sufficiently powerful to implement a variety of updating policies. We have formalized the core of ReCaml and proved it sound (using the Coq proof assistant), thus ensuring that state-manipulating updates preserve type-safe execution of the updated program. We have implemented ReCaml as an extension to the Caml bytecode interpreter and used it for several examples.

Templates-based portable just-in-time compiler

Usage of the platform-neutral bytecode interpreters is often limited by their restricted performance. Just-in-time compilers effectively solve this problem. However they are hard to develop and retarget. This paper demonstrates that dynamic code generation from the templates created by a C compiler can be used to build a simple and highly-portable JIT compiler.

A compiled implementation of strong reduction

Motivated by applications to proof assistants based on dependent types, we develop and prove correct a strong reducer and ß-equivalence checker for the λ-calculus with products, sums, and guarded fixpoints. Our approach is based on compilation to the bytecode of an abstract machine performing weak reductions on non-closed terms, derived with minimal modifications from the ZAM machine used in the Objective Caml bytecode interpreter, and complemented by a recursive "read back" procedure. An implementation in the Coq proof assistant demonstrates important speed-ups compared with the original interpreter-based implementation of strong reduction in Coq.

Bytecode compression via profiled grammar rewriting

This paper describes the design and implementation of a method for producing compact, bytecoded instruction sets and interpreters for them. It accepts a grammar for programs written using a simple bytecoded stack-based instruction set, as well as a training set of sample programs. The system transforms the grammar, creating an expanded grammar that represents the same language as the original grammar, but permits a shorter derivation of the sample programs and others like them. A program's derivation under the expanded grammar forms the compressed bytecode representation of the program. The interpreter for this bytecode is automatically generated from the original bytecode interpreter and the expanded grammar. Programs expressed using compressed bytecode can be substantially smaller than their original bytecode representation and even their machine code representation. For example, compression cuts the bytecode for lcc from 199KB to 58KB but increases the size of the interpreter by just over 11KB.

A quantitative study for Java microprocessor architectural requirements. Part I: Instruction set design

Java was designed for network programming. This imposes certain requirements on its virtual machine instruction set architecture and on designs that support Java. The purpose of this study is to carry out a behavioral analysis of the different aspects of Java instruction set architecture. This will help in establishing the hardware requirements for executing Java bytecodes. First, the bytecode interpreter was instrumented to include performance counters and statistics collectors. Then performance metrics were collected through benchmarking. Analyzing these data helps to identify performance-critical aspects that are candidates for hardware support, while less critical aspects can be left for software implementations. In this first part of our two-part paper, we study access patterns for data types, addressing modes, and instruction encoding. Recommendations for architectural requirements for Java processors will be made throughout this study.

A quantitative study for Java microprocessor architectural requirements. Part II: high-level language support

Java was designed for network programming. This imposes certain requirements on its virtual machine instruction set architecture and on designs that intend to support Java. The purpose of this study is to carry out a behavioral analysis of the different aspects of Java instruction set architecture. Performance metrics were collected through benchmarking a bytecode interpreter. In this second part of our two-part paper, we study the instruction set utilization, instruction execution time, method invocation behavior, and the effect of object-orientation.

Static and Dynamic Program Compilation by Interpreter Specialization

Interpretation and run-time compilation techniques are increasingly important because they can support heterogeneous architectures, evolving programming languages, and dynamically-loaded code. Interpretation is simple to implement, but yields poor performance. Run-time compilation yields better performance, but is costly to implement. One way to preserve simplicity but obtain good performance is to apply program specialization to an interpreter in order to generate an efficient implementation of the program automatically. Such specialization can be carried out at both compile time and run time. Recent advances in program-specialization technology have significantly improved the performance of specialized interpreters. This paper presents and assesses experiments applying program specialization to both bytecode and structured-language interpreters. The results show that for some general-purpose bytecode languages, specialization of an interpreter can yield speedups of up to a factor of four, while specializing certain structured-language interpreters can yield performance comparable to that of an implementation in a general-purpose language, compiled using an optimizing compiler.

Compiling lazy functional programs for the Java Virtual Machine

In this paper, we show how lazy functional programs can be compiled for the Java Virtual Machine using a mapping between a version of the 〈v, G〉-machine and the Java Virtual Machine. This mapping is elegant – the description is entirely straightforward – and efficient – using it, both code size and execution speed are of the same order of magnitude as those obtained with a traditional functional language bytecode interpreter. In future, our work could serve as the basis of an interface between Haskell and Java.