Incremental Compilation Research Articles

Parallelising semantic checking in an IDE: A way toward improving profits and sustainability, while maintaining high-quality software development

AbstractAfter recent improvements brought the incremental compilation of large industrial test suites down to a few seconds, the first semantic checking of a project became one of the longest-running processes. As multi-core systems are now the standard, we derived a parallelisation using software engineering laws to improve the performance of semantic checking.Our measurements show that even an outdated laptop is fast enough for daily use. The performance improvements came without performance regressions, and we can’t expect additional massive benefits even from infinitely scaling Cloud resources.Companies should utilise cheaper machines that still o er enough performance for longer. This approach can help businesses increase profits, reduce electronic waste and promote sustainability while maintaining high-quality software development practices.

ExHiPR: Extended High-level Partial Reconfiguration for Fast Incremental FPGA Compilation

Partial Reconfiguration (PR) is a key technique in the application design on modern FPGAs. However, current PR tools heavily rely on the developer to manually conduct PR module definition, floorplanning, and flow control at a low level. The existing PR tools do not consider High-Level-Synthesis languages either, which are of great interest to software developers. We propose HiPR, an open-source framework, to bridge the gap between HLS and PR. HiPR allows the developer to define partially reconfigurable C/C++ functions, instead of Verilog modules, to accelerate the FPGA incremental compilation and automate the flow from C/C++ to bitstreams. We use a lightweight Simulated Annealing floorplanner and show that it can produce high-quality PR floorplans an order of magnitude faster than analytic methods. By mapping Rosetta HLS benchmarks, we demonstrate that the incremental compilation can be accelerated by 3–10 × compared with state-of-the-art Xilinx Vitis flow without performance loss, at the cost of 15-67% one-time overlay set-up time.

Constructing Hybrid Incremental Compilers for Cross-Module Extensibility with an Internal Build System

Context: Compilation time is an important factor in the adaptability of a software project. Fast recompilation enables cheap experimentation with changes to a project, as those changes can be tested quickly. Separate and incremental compilation has been a topic of interest for a long time to facilitate fast recompilation. Inquiry: Despite the benefits of an incremental compiler, such compilers are usually not the default. This is because incrementalization requires cross-cutting, complicated, and error-prone techniques such as dependency tracking, caching, cache invalidation, and change detection. Especially in compilers for languages with cross-module definitions and integration, correctly and efficiently implementing an incremental compiler can be a challenge. Retrofitting incrementality into a compiler is even harder. We address this problem by developing a compiler design approach that reuses parts of an existing non-incremental compiler to lower the cost of building an incremental compiler. It also gives an intuition into compiling difficult-to-incrementalize language features through staging. Approach: We use the compiler design approach presented in this paper to develop an incremental compiler for the Stratego term-rewriting language. This language has a set of features that at first glance look incompatible with incremental compilation. Therefore, we treat Stratego as our critical case to demonstrate the approach on. We show how this approach decomposes the original compiler and has a solution to compile Stratego incrementally. The key idea on which we build our incremental compiler is to internally use an incremental build system to wire together the components we extract from the original compiler. Knowledge: The resulting compiler is already in use as a replacement of the original whole-program compiler. We find that the incremental build system inside the compiler is a crucial component of our approach. This allows a compiler writer to think in multiple steps of compilation, and combine that into a incremental compiler almost effortlessly. Normally, separate compilation \`a la C is facilitated by an external build system, where the programmer is responsible for managing dependencies between files. We reuse an existing sound and optimal incremental build system, and integrate its dependency tracking into the compiler. Grounding: The incremental compiler for Stratego is available as an artefact along with this article. We evaluate it on a large Stratego project to test its performance. The benchmark replays edits to the Stratego project from version control. These benchmarks are part of the artefact, packaged as a virtual machine image for easy reproducibility. Importance: Although we demonstrate our design approach on the Stratego programming language, we also describe it generally throughout this paper. Many currently used programming languages have a compiler that is much slower than necessary. Our design provides an approach to change this, by reusing an existing compiler and making it incremental within a reasonable amount of time.

Incremental evaluation of higher-order attributes

Compilers, amongst other programs, often work with data that (slowly) changes over time. When the changes between subsequent runs of the compiler are small, one would hope the compiler to incrementally update its results, resulting in much lower rebuilding times. However, the manual construction of an incremental compiler is very hard and error prone and therefore usually not an option.

Design of methodology for incremental compiler construction

The paper deals with possibilities of the incremental compiler construction. It represents the compiler construction possibilities for languages with a fixed set of lexical units and for languages with a variable set of lexical units, too. The methodology design for the incremental compiler construction is based on the known algorithms for standard compiler construction and derived for both groups of languages. Under the group of languages with a fixed set of lexical units there belong languages, where each lexical unit has its constant meaning, e.g., common programming languages. For this group of languages the paper tries to solve the problem of the incremental semantic analysis, which is based on incremental parsing. In the group of languages with a variable set of lexical units (e.g., professional typographic system TEX), it is possible to change arbitrarily the meaning of each character on the input file at any time during processing. The change takes effect immediately and its validity can be somehow limited or is given by the end of the input. For this group of languages this paper tries to solve the problem case when we use macros temporarily changing the category of arbitrary characters.

Declarative Consciousness for Reconstruction

Abstract Existing information technology tools are harnessed and integrated to provide digital specification of human consciousness of individual persons. An incremental compilation technology is proposed as a transformation of LifeLog derived persona specifications into a Canonical representation of the neocortex architecture of the human brain. The primary purpose is to gain an understanding of the semantical allocation of the neocortex capacity. Novel neocortex content allocation simulators with browsers are proposed to experiment with various approaches of relieving the brain from overload conditions. An IT model of the neocortex is maintained, which is then updated each time new stimuli are received from the LifeLog data stream; new information is gained from brain signal measurements; and new functional dependencies are discovered between live persona consumed/produced signals

Incremental Knowledge Compilation Using the Extension Rule

Knowledge compilation by using the extension rule (KCER) has been recognized as a novel compilation approach, although this method can only deal with static knowledge bases. This paper proposes a novel incremental knowledge compilation method by using the extension rule so that it can tackle knowledge compilation problems in the dynamic environment. The method does not recompile the whole knowledge base, while it is to use the information of compiling original knowledge base to speed up the compiling process. The experimental results show that this method can deal with dynamic knowledge bases efficiently

A model‐based framework for building extensible, high performance stream processing middleware and programming language for IBM InfoSphere Streams

SUMMARYThis work presents an extensive case study on the model‐based design of a commercial‐grade stream processing middleware (IBM's InfoSphere Streams) its runtime and language (SPL) compiler. The model‐based underpinnings are pervasive throughout the whole environment, from describing inter‐process communication interfaces and objects to the design of the extensibility mechanism in the runtime and language. In addition to many software engineering advantages such as consistent, uniform, and self‐documented integration among the different parts of the system, we show intrinsic performance benefits to the platform derived from this design approach. First, we demonstrate how an incremental compilation strategy employed by the SPL compiler and rooted on the model description of the application, extracted by the compiler as part of the application building process, leads to better compile‐time performance. Second, we discuss how the model‐based code generation strategy employed by the SPL compiler also leads to increased runtime performance, by specializing the generated code to particular characteristics of the runtime environment. Finally, we show how the extensibility strategy used in the SPL language leads to automatic syntactic and semantic checks at compile time, while enabling behavioral reasoning and specific optimizations at runtime. Copyright © 2011 John Wiley & Sons, Ltd.

INCREMENTAL COMPILATION OF BAYESIAN NETWORKS BASED ON MAXIMAL PRIME SUBGRAPHS

When a Bayesian network (BN) is modified, for example adding or deleting a node, or changing the probability distributions, we usually will need a total recompilation of the model, despite feeling that a partial (re)compilation could have been enough. Especially when considering dynamic models, in which variables are added and removed very frequently, these recompilations are quite resource consuming. But even further, for the task of building a model, which is in many occasions an iterative process, there is a clear lack of flexibility. When we use the term Incremental Compilation or IC we refer to the possibility of modifying a network and avoiding a complete recompilation to obtain the new (and different) join tree (JT). The main point we intend to study in this work is JT-based inference in Bayesian networks. Apart from undertaking the triangulation problem itself, we have achieved a great improvement for the compilation in BNs. We do not develop a new architecture for BNs inference, but taking some already existing framework JT-based for probability propagation such as Hugin or Shenoy and Shafer, we have designed a method that can be successfully applied to get better performance, as the experimental evaluation will show.

Average-case analysis of incremental topological ordering

Many applications like pointer analysis and incremental compilation require maintaining a topological ordering of the nodes of a directed acyclic graph (DAG) under dynamic updates. All known algorithms for this problem are either only analyzed for worst-case insertion sequences or only evaluated experimentally on random DAGs. We present the first average-case analysis of incremental topological ordering algorithms. We prove an expected runtime of O ( n 2 polylog ( n ) ) under insertion of the edges of a complete DAG in a random order for the algorithms of Alpern et al. (1990) [4], Katriel and Bodlaender (2006) [18], and Pearce and Kelly (2006) [23].

Contract Specification in Java: Classification, Characterization, and a New Marker Method

Design by Contract (DBC), originated in the Eiffel programming language, is generally accepted as a practical method for building reliable software. Currently, however, few languages have built-in support for it. In recent years, several methods have been proposed to support DBC in Java. We compare eleven DBC tools for Java by analyzing their impact on the developer's programming activities, which are characterized by seven quality attributes identified in this paper. It is shown that each of the existing tools fails to achieve some of the quality attributes. This motivates us to develop ezContract, an open source DBC tool for Java that achieves all of the seven quality attributes. ezContract achieves streamlined integration with the working environment. Notably, standard Java language is used and advanced IDE features that work for standard Java programs can also work for the contract-enabled programs. Such features include incremental compilation, automatic refactoring, and code assist.

Nu

For a number of reasons, such as to generate object code that is compliant with the existing virtual machines (VM), current compilers for aspect-oriented languages sacrifice design modularity when transforming source to object code by losing textual locality and intermingling concerns in the object code. Sacrificing design modularity has significant costs, especially in terms of the speed of incremental compilation. We present an intermediate language design that preserves aspect-oriented design modularity in Java byte code. We briefly describe our extensions to the Sun Hotspot VM to support the new intermediate language design.

A first-class approach to genericity

This paper describes how to add first-class generic types---including mixins---to strongly-typed OO languages with nominal subtyping such as Java and C#. A generic type system is "first-class" if generic types can appear in any context where conventional types can appear. In this context, a mixin is simply a generic class that extends one of its type parameters, e.g. , a class C<T> that extends T. Although mixins of this form are widely used in Cpp (via templates), they are clumsy and error-prone because Cpp treats mixins as macros, forcing each mixin instantiation to be separately compiled and type-checked. The abstraction embodied in a mixin is never separately analyzed.Our formulation of mixins using first-class genericity accommodates sound local (class-by-class) type checking. A mixin can be fully type-checked given symbol tables for each of the classes that it directly references---the same context in which Java performs incremental class compilation. To our knowledge, no previous formal analysis of first-class genericity in languages with nominal type systems has been conducted, which is surprising because nominal subtyping has become predominant in mainstream object-oriented programming languages.What makes our treatment of first-class genericity particularly interesting and important is the fact that it can be added to the existing Java language without any change to the underlying Java Virtual Machine. Moreover, the extension is backward compatible with legacy Java source and class files. Although our discussion of a practical implementation strategy focuses on Java, the same implementation techniques could be applied to other object-oriented languages such as C# or Eiffel that support incremental compilation, dynamic class loading, and nominal subtyping.

Incremental compilation for parallel logic verification systems

Although simulation remains an important part of application-specific integrated circuit (ASIC) validation, hardware-assisted parallel verification is becoming a larger part of the overall ASIC verification flow. In this paper, we describe and analyze a set of incremental compilation steps that can be directly applied to a range of parallel logic verification hardware, including logic emulators. Important aspects of this work include the formulation and analysis of two incremental design mapping steps: the partitioning of newly added design logic onto multiple logic processors and the communication scheduling of newly added design signals between logic processors. To validate our incremental compilation techniques, the developed mapping heuristics have been integrated into the compilation flow for a field-programmable gate-array-based Ikos VirtuaLogic emulator . The modified compiler has been applied to five large benchmark circuits that have been synthesized from register-transfer level and mapped to the emulator. It is shown that our incremental approach reduces verification compile time for modified designs by up to a factor of five versus complete design recompilation for benchmarks of over 100 000 gates. In most cases, verification run-time following incremental compilation of a modified design matches the performance achieved with complete design recompilation.

The Ciao Modular, Standalone Compiler and Its Generic Program Processing Library

Ciao Prolog incorporates a module system which allows separate compilation and sensible creation of standalone executables. We describe some of the main aspects of the Ciao modular compiler, ciaoc, which takes advantage of the characteristics of the Ciao Prolog module system to automatically perform separate and incremental compilation and efficiently build small, standalone executables with competitive run-time performance, ciaoc can also detect statically a larger number of programming errors. We also present a generic code processing library for handling modular programs, which provides an important part of the functionality of ciaoc. This library allows the development of program analysis and transformation tools in a way that is to some extent orthogonal to the details of module system design, and has been used in the implementation of ciaoc and other Ciao system tools. We also describe the different types of executables which can be generated by the Ciao compiler, which offer different tradeoffs between executable size, startup time, and portability, depending, among other factors, on the linking regime used (static, dynamic, lazy, etc.). Finally, we provide experimental data which illustrate these tradeoffs.

Design and implementation of a client‐server architecture for taxonomy manager

Taxonomy Manager (TM) is a computer-based, full-text method dedicated to represent biological knowledge allowing scientists to continuously revise and reorganize the conceptual framework of data and their interpretation. The system architecture distinguishes clients and a task oriented server. TM provides a distinct, precise, yet user-friendly, formalism for knowledge specification by biologic experts and for factual knowledge input by general scientists on interactive interfaces. An incremental compiler translates and transforms new information specified within full-text editors into an internal format and back again. A retrieval dialog allows access to the results of text analyses including information units and associated rules, and a browser offers free navigation through the network of data. External users can access the information contained within TM by a communication protocol. Copyright © 1999 John Wiley & Sons, Ltd.

The architecture of Montana

Montana is an open, extensible integrated programming environment for C++ that supports incremental compilation and linking, a persistent code cache called a CodeStore, and a set of programming interfaces to the CodeStore for tool writers. CodeStore serves as a central source of information for compiling, browsing, and debugging. CodeStore contains information about both the static and dynamic structure of the compiled program. This information spans files, macros, declarations, function bodies, templates and their instantiations, program fragment dependencies, linker relocation information, and debugging information.Montana allows the compilation process to be extended and modified [11]. Montana has been used as the basis of a number of tools [1,7], and is also used as the infrastructure of a production compiler, IBM's Visual Age C++ 4.0 [8].

Achieving Incremental Compilation through Fine-grained Builds

Achieving Incremental Compilation through Fine-grained Builds

Achieving Incremental Compilation through Fine‐grained Builds

Achieving Incremental Compilation through Fine‐grained Builds

A List‐based Virtual Machine for COBOL

The concepts of COBOL can be well represented by a virtual machine that makes use of descriptors and descriptor lists. The architecture of such virtual machine for COBOL, which supports incremental compilation, is presented in this paper. The structure of different descriptors and descriptor lists of the virtual machine is described. Principles of incremental compilation of COBOL paragraphs are given. The use of stacks for holding temporary values and subprogram parameters is explained. Addressing modes of the virtual machine are discussed, and its instruction set is presented. Options for efficient implementation of the virtual machine on a real machine are discussed.