Machine Code Generation Research Articles

Three etudes on compiling, or How to use the Julia language to study generated machine code

The article demonstrates the ability of the Julia programming language to output an assembly text for any function written in Julia, which is interesting for education. The executable code for three illustrative examples is examined in detail: calculation by formula, branching and cyclic fragments. It is shown that Julia compiler really generates a carefully optimized program, trying to use the fastest executing instructions of the Intel processor. It successfully tries to do without time-consuming arithmetic operations (if possible, the values are calculated in advance, multiplication is replaced by shift and so on). The results for integer data are especially impressive. It is possible to see a whole range of optimization techniques, including those that take into account the features of the algorithm itself: for example, the compiler is able to predict the results of the simplest loop and exclude this loop from the final program. The problem of overflow control in a program is also discussed.The consideration carried out convincingly confirms the usefulness of theoretical knowledge for developing an effectively working program. The described analysis technique is interesting for education, because the importance of a deep understanding by modern specialists of the programming process essence cannot be overestimated. The material can be useful in teaching programming ideas at different levels. The detailed presentation makes the article readable without special preparation.

Efficient Compiler Design for a Geometric Shape Domain-Specific Language: Emphasizing Abstraction and Optimization Techniques

The research paper represents a novel approach to the design and optimization of a compiler for a domain-specific language (DSL) focused on geometric shape creation and manipulation. The primary objective is to develop a compiler capable of generating efficient machine code while offering users a high level of abstraction. The paper begins with an overview of DSLs and compilers, emphasizing their importance in software development. Next, it outlines the specific requirements of the geometric shape DSL and proposes a compiler design that addresses them. This innovative approach considers DSL's unique features, such as shape creation and manipulation, and aims to generate high-quality machine code. The paper also discusses optimization techniques to enhance the generated code's quality and performance, including loop unrolling and instruction scheduling. These optimizations are particularly suited to the DSL, which focuses on geometric shape creation and manipulation and are integral to achieving efficient machine code generation. In conclusion, the paper emphasizes the novelty of this approach to DSL compiler design and anticipates exciting results from testing the compiler developed for the geometric shape DSL.

Manufacturability-Based Design Optimization for Directed Energy Deposition Processes

Additive Manufacturing (AM) is the process of joining materials by selectively depositing them layer upon layer for the purpose of manufacturing parts or assemblies which are based on a 3D digital model. The nature of these processes results in the morphing of complex component geometries, enabling a high degree of design freedom and resulting in lightweight structures with increased performance. These processes, however, experience many limitations regarding manufacturability. The aim of this study is to develop a method and tool that optimizes the design of a component to avoid overhanging geometries and the need for supports during the Additive Manufacturing process. A workflow consisting of steps for topology optimization, orientation optimization, material addition, and machine code generation is described and implemented using Rhinoceros 3D and Grasshopper software. The proposed workflow is compared to a conventional workflow regarding manufacturing Key Performance Indicators (KPIs) such as part volume, support volume, and build time. A significant reduction is observed regarding all the KPIs by using the proposed method. Examining the results from both the conventional workflow and the proposed one, it is clear that the latter has unquestionable advantages in terms of effectiveness. In the particular case study presented, a total volume reduction of around 80% is observed. The reduction in the total volume (including the required support volume) leads to a significant reduction in the material used as well as in the build time, consequently resulting in cost reduction.

AMOM: Adaptive Masking over Masking for Conditional Masked Language Model

Transformer-based autoregressive (AR) methods have achieved appealing performance for varied sequence-to-sequence generation tasks, e.g., neural machine translation, summarization, and code generation, but suffer from low inference efficiency. To speed up the inference stage, many non-autoregressive (NAR) strategies have been proposed in the past few years. Among them, the conditional masked language model (CMLM) is one of the most versatile frameworks, as it can support many different sequence generation scenarios and achieve very competitive performance on these tasks. In this paper, we further introduce a simple yet effective adaptive masking over masking strategy to enhance the refinement capability of the decoder and make the encoder optimization easier. Experiments on 3 different tasks (neural machine translation, summarization, and code generation) with 15 datasets in total confirm that our proposed simple method achieves significant performance improvement over the strong CMLM model. Surprisingly, our proposed model yields state-of-the-art performance on neural machine translation (34.62 BLEU on WMT16 EN to RO, 34.82 BLEU on WMT16 RO to EN, and 34.84 BLEU on IWSLT De to En) and even better performance than the AR Transformer on 7 benchmark datasets with at least 2.2x speedup. Our code is available at GitHub.

PartitionTuner: An operator scheduler for deep‐learning compilers supporting multiple heterogeneous processing units

AbstractRecently, embedded systems, such as mobile platforms, have multiple processing units that can operate in parallel, such as centralized processing units (CPUs) and neural processing units (NPUs). We can use deep‐learning compilers to generate machine code optimized for these embedded systems from a deep neural network (DNN). However, the deep‐learning compilers proposed so far generate codes that sequentially execute DNN operators on a single processing unit or parallel codes for graphic processing units (GPUs). In this study, we propose PartitionTuner, an operator scheduler for deep‐learning compilers that supports multiple heterogeneous PUs including CPUs and NPUs. PartitionTuner can generate an operator‐scheduling plan that uses all available PUs simultaneously to minimize overall DNN inference time. Operator scheduling is based on the analysis of DNN architecture and the performance profiles of individual and group operators measured on heterogeneous processing units. By the experiments for seven DNNs, PartitionTuner generates scheduling plans that perform 5.03% better than a static type‐based operator‐scheduling technique for SqueezeNet. In addition, PartitionTuner outperforms recent profiling‐based operator‐scheduling techniques for ResNet50, ResNet18, and SqueezeNet by 7.18%, 5.36%, and 2.73%, respectively.

Recursive SQL and GPU-support for in-database machine learning

In machine learning, continuously retraining a model guarantees accurate predictions based on the latest data as training input. But to retrieve the latest data from a database, time-consuming extraction is necessary as database systems have rarely been used for operations such as matrix algebra and gradient descent. In this work, we demonstrate that SQL with recursive tables makes it possible to express a complete machine learning pipeline out of data preprocessing, model training and its validation. To facilitate the specification of loss functions, we extend the code-generating database system Umbra by an operator for automatic differentiation for use within recursive tables: With the loss function expressed in SQL as a lambda function, Umbra generates machine code for each partial derivative. We further use automatic differentiation for a dedicated gradient descent operator, which generates LLVM code to train a user-specified model on GPUs. We fine-tune GPU kernels at hardware level to allow a higher throughput and propose non-blocking synchronisation of multiple units. In our evaluation, automatic differentiation accelerated the runtime by the number of cached subexpressions compared to compiling each derivative separately. Our GPU kernels with independent models allowed maximal throughput even for small batch sizes, making machine learning pipelines within SQL more competitive.

Development and implementation of a software for wire arc additive manufacturing preprocessing planning: trajectory planning and machine code generation

To overcome a shortage of flexible and low-cost solutions for wire arc additive manufacturing (WAAM) preprocessing, this work’s objective was to develop and validate an in-house computational programme in an open-source environment for WAAM preprocessing planning. Algorithms for reading STL (stereolithography) files and implementing rotation, slicing, trajectory planning, and machine code generation were elaborated and implemented in the Scilab environment (free and open-source). A graphical interface was developed to facilitate user interaction, with 5 options for path planning. The functionality of each work step is detailed. For validation of the software, single and multiple-layer prints, with different geometrical complexity and printing challenges, were built in a CNC table geared by the generated machine code. The validation criteria were deposition imperfection, morphological, and dimensional tolerances. The outputs showed that the parts were successfully printed. Therefore, this work demonstrates that Scilab provides the necessary resources for companies and universities to implement and/or develop algorithms for planning and generating trajectories for WAAM. Moreover, emerging ideas can be reasonably easily implemented in such software, not always possible in commercial packages.

Reconciling optimization with secure compilation

Software protections against side-channel and physical attacks are essential to the development of secure applications. Such protections are meaningful at machine code or micro-architectural level, but they typically do not carry observable semantics at source level. This renders them susceptible to miscompilation, and security engineers embed input/output side-effects to prevent optimizing compilers from altering them. Yet these side-effects are error-prone and compiler-dependent. The current practice involves analyzing the generated machine code to make sure security or privacy properties are still enforced. These side-effects may also be too expensive in fine-grained protections such as control-flow integrity. We introduce observations of the program state that are intrinsic to the correct execution of security protections, along with means to specify and preserve observations across the compilation flow. Such observations complement the input/output semantics-preservation contract of compilers. We introduce an opacification mechanism to preserve and enforce a partial ordering of observations. This approach is compatible with a production compiler and does not incur any modification to its optimization passes. We validate the effectiveness and performance of our approach on a range of benchmarks, expressing the secure compilation of these applications in terms of observations to be made at specific program points.

Words to Matter: De novo Architected Materials Design Using Transformer Neural Networks

Transformer neural networks have become widely used in a variety of AI applications, enabling significant advances in Natural Language Processing (NLP) and computer vision. Here we demonstrate the use of transformer neural networks in the de novo design of architected materials using a unique approach based on text input that enables the design to be directed by descriptive text, such as “a regular lattice of steel”. Since transformer neural nets enable the conversion of data from distinct forms into one another, including text into images, such methods have the potential to be used as a natural-language-driven tool to develop complex materials designs. In this study we use the Contrastive Language-Image Pre-Training (CLIP) and VQGAN neural networks in an iterative process to generate images that reflect text prompt driven materials designs. We then use the resulting images to generate three-dimensional models that can be realized using additive manufacturing, resulting in physical samples of these text-based materials. We present several such word-to-matter examples, and analyze 3D printed material specimen through associated additional finite element analysis, especially focused on mechanical properties including mechanism design. As an emerging new field, such language-based design approaches can have profound impact, including the use of transformer neural nets to generate machine code for 3D printing, optimization of processing conditions, and other end-to-end design environments that intersect directly with human language.

ComVIS—Interactive simulation environment for compiler learning

Simulation-based learning tools have become commonly used at all levels of education. They are recognized as an effective means of studying complex and abstract systems. In traditional teaching of compiler concepts, students are incapable of visualizing theoretical constructions, while the use of simulation systems represents an excellent blend of theoretical and practical experience. This paper describes a software system for visualization and simulation of basic compiler concepts and algorithms. The simulation tool enables students to visualize the operation of finite automata, convert a regular expression into deterministic finite automata (DFA) or nondeterministic finite automata, and simulate Thompson's construction algorithm. The process of constructing the LL(0) and LR(1) parsing tables is represented as a step-by-step simulation, with the user receives feedback on the correctness of each step before moving on to the next. The system is capable of calculating the FIRST and FOLLOW sets, graphically representing DFA, and constructing the corresponding parsing table. This tool provides a module for visualizing the process of machine code generation.

Do you have space for dessert? a verified space cost semantics for CakeML programs

Garbage collectors relieve the programmer from manual memory management, but lead to compiler-generated machine code that can behave differently (e.g. out-of-memory errors) from the source code. To ensure that the generated code behaves exactly like the source code, programmers need a way to answer questions of the form: what is a sufficient amount of memory for my program to never reach an out-of-memory error? This paper develops a cost semantics that can answer such questions for CakeML programs. The work described in this paper is the first to be able to answer such questions with proofs in the context of a language that depends on garbage collection. We demonstrate that positive answers can be used to transfer liveness results proved for the source code to liveness guarantees about the generated machine code. Without guarantees about space usage, only safety results can be transferred from source to machine code. Our cost semantics is phrased in terms of an abstract intermediate language of the CakeML compiler, but results proved at that level map directly to the space cost of the compiler-generated machine code. All of the work described in this paper has been developed in the HOL4 theorem prover.

Live modeling in the context of state machine models and code generation

Live modeling has been recognized as an important technique to edit behavioral models while being executed and helps in better understanding the impact of a design choice. In the context of Model-driven Development (MDD) models can be executed by interpretation or by the translation of models into existing programming languages, often by code generation. This work is concerned with the support of live modeling in the context of state machine models when they are executed by code generation. To this end, we propose an approach that is completely independent of any live programming support offered by the target language. This independence is achieved with the help of a model transformation which equips the model with support for features which are required for live modeling. A subsequent code generation then produces a self-reflective program that allows changes to the model elements at runtime (through synchronization of design and runtime models). We have applied the approach in the context of UML-RT and created a prototype (Live-UMLRT) that provides a full set of services for live modeling of UML-RT state machines such as re-execution, adding/removing states and transitions, and adding/removing action code. We have evaluated the prototype on several use-cases. The evaluation shows that (1) generation of a self-reflective and model instrumentation can be carried out with reasonable performance, and (2) our approach can apply model changes to the running execution faster than the standard approach that depends on the live programming support of the target language.

Кэширование машинного кода в динамическом компиляторе SQL-запросов для СУБД PostgreSQL

As the efficiency of main and external memory grows, alongside with decreasing hardware costs, the performance of database management systems (DBMS) on certain kinds of queries is more determined by CPU characteristics and the way it is utilized. Relational DBMS utilize diverse execution models to run SQL queries. Those models have different properties, but in either way suffer from substantial overhead during query plan interpretation. The overhead comes from indirect calls to handler functions, runtime checks and large number of branch instructions. One way to solve this problem is dynamic query compilation that is reasonable only in those cases when query interpretation time is larger than the time of compilation and optimized machine code execution. This requirement can be satisfied only when the amount of data to be processed is large enough. If query interpretation takes milliseconds to finish, then the cost of dynamic compilation can be hundreds of times more than the execution time of generated machine code. To pay off the cost of dynamic compilation, the generated machine code has to be reused in subsequent executions, thus saving the cost of code compilation and optimization. In this paper, we examine the method of machine code caching in our query JIT-compiler for DBMS PostgreSQL. The proposed method allows us to eliminate compilation overhead. The results show that dynamic compilation of queries with machine code caching feature gives a significant speedup on OLTP queries.

Genetic Programming-Based Code Generation for Arduino

This article describes a methodology for writing the program for the Arduino board using an automatic generator of assembly language routines that works based on a cooperative coevolutionary multi-objective linear genetic programming algorithm. The methodology is described in an illustrative example that consists of the development of the program for a digital thermometer organized on a circuit formed by the Arduino Mega board, a text LCD module, and a temperature sensor. The automatic generation of a routine starts with an input-output table that can be created in a spreadsheet. The following routines have been automatically generated: initialization routine for the text LCD screen, routine for determining the temperature value, routine for converting natural binary code into unpacked two-digit BCD code, routine for displaying a symbol on the LCD screen. The application of this methodology requires basic knowledge of the assembly programming language for writing the main program and some initial configuration routines. With the application of this methodology in the illustrative example, 27% of the program lines were written manually, while the remaining 73% were generated automatically. The program, produced with the application of this methodology, preserves the advantage of assembly language programs of generating machine code much smaller than that generated by using the Arduino programming language.

An optimal method for enhancing the generation of machine code from natural language data set

Natural language processing is a very active area of research and development, there is not a single agreed upon a method that would satisfy everyone for the use of natural language to operate electronic devices or other practical applications. But there are some aspects used from many years in the formulation and solution of computational problem arising in natural language processing. This paper describes a model in which numerical values are assigned to word of natural language speech data set to convert the information present in natural language speech data set into an intermediate numeric form as a structured data set. The intermediated numerical values of each word will be used for generation of machine code which will be easily understand by electronic devices to draw inferences from data set. The designed model is useful for a number of practical applications and very simple to implement.

Path calculation of 7-axes synchronous quasi-tangential laser manufacturing

Quasi-tangential laser processing, also called laser turning, is increasingly applied for various applications. Specifically, its ability to generate complex geometries with small feature sizes at high precision and surface quality in hard, brittle, and electrically non-conductive materials is a key benefit. Due to the geometric flexibility, the process is well suited for prototyping in hard-to-machine materials such as ceramics, carbides, and super-abrasives. However, the lack of advanced software solutions for this novel process hitherto limited the exploitation of the potential. Here, we discuss a unique computer-aided manufacturing approach for synchronous 7-axes laser manufacturing with quasi-tangential strategies. This gives the peerless possibility to process arbitrary geometries, which cannot be manufactured with conventional techniques. A detailed description of the path calculation with derivation and procedures is given. The generated machine code is tested on a laser manufacturing setup consisting of five mechanical and two optical axes. Following, a processed cylindrical ceramic specimen with a continuously varying profile along a helical path is presented. The profile is constituted by a rectangular over half-spherical to a triangular groove with defined pitch on the helix. This demonstrator provides the validation of the presented CAM solution. Measurements of the produced specimen show high adherence with the target geometry and an average deviation below 10 μm.

The verified CakeML compiler backend

AbstractThe CakeML compiler is, to the best of our knowledge, the most realistic verified compiler for a functional programming language to date. The architecture of the compiler, a sequence of intermediate languages through which high-level features are compiled away incrementally, enables verification of each compilation pass at an appropriate level of semantic detail. Parts of the compiler’s implementation resemble mainstream (unverified) compilers for strict functional languages, and it supports several important features and optimisations. These include efficient curried multi-argument functions, configurable data representations, efficient exceptions, register allocation, and more. The compiler produces machine code for five architectures: x86-64, ARMv6, ARMv8, MIPS-64, and RISC-V. The generated machine code contains the verified runtime system which includes a verified generational copying garbage collector and a verified arbitrary precision arithmetic (bignum) library. In this paper, we present the overall design of the compiler backend, including its 12 intermediate languages. We explain how the semantics and proofs fit together and provide detail on how the compiler has been bootstrapped inside the logic of a theorem prover. The entire development has been carried out within the HOL4 theorem prover.

Making Compiling Query Engines Practical

Compiling queries to machine code is a very efficient way for executing queries. One often overlooked problem with compilation is the time it takes to generate machine code. Even with fast compilation frameworks like LLVM, generating machine code for complex queries often takes hundreds of milliseconds. Such durations can be a major disadvantage for workloads that execute many complex, but quick queries. To solve this problem, we propose an adaptive execution framework, which dynamically switches from interpretation to compilation. We also propose a fast bytecode interpreter for LLVM, which can execute queries without costly translation to machine code and dramatically reduces the query latency. Adaptive execution is fine-grained, and can execute code paths of the same query using different execution modes. Our evaluation shows that this approach achieves optimal performance in a wide variety of settings—low latency for small data sets and maximum throughput for large data sizes. Besides compilation time, we also focus on debugging, which is another important challenge of compilation-based query engines. To address this problem, we present a novel, database-specific debugger for compiling query engines.

Applying High-Level Function Loop Invariants for Machine Code Deductive Verification

The existing tools of deductive verification allow us to successfully prove the correctness of functions written in high-level languages such as C or Java. However, this may not be enough for critical software because even fully verified code cannot guarantee the correct generation of machine code by the compiler. At the moment, developers of such systems have to accept the compiler correctness assumption, which, however, is extremely undesirable, but inevitable due to the lack of full-fledged systems of formal verification of machine code. It is also worth noting that the verification of machine code by a person directly is an extremely time-consuming task due to the high complexity and large amounts of machine code. One of the approaches to simplify the verification of machine code is automatic deductive verification reusing the formal specification of the high-level language function. The formal specification of the function consists of the specification of the pre- and postcondition, as well as loop invariants, which specify conditions that are satified at each iteration of the loop. When compiling a program into machine code, pre- and postconditions are preserved, which, however, cannot be said about loop invariants. This fact is one of the main problems of automatic verification of machine code with loops. Another important problem is that high-level function variables often have ‘projections' to both registers and memory at the machine code level, and the verification procedure requires that invariants be described for each variable, and therefore the missing invariants must be generated. This paper presents an approach to solving these problems, based on the analysis of the control flow graph, and intelligent search of the locations of variables.

Совмещение ACSL спецификаций с машинным кодом

When developing programs in high-level languages, developers have to make assumptions about the correctness of the compiler. However, this may be unacceptable for critical systems. As long as there are no full-fledged formally verified compilers, the author proposes to solve this problem by proving the correctness of the generated machine code by deductive verification. To achieve this goal, it is required to combine the pre- and postcondition specifications with the machine code behavior model. The paper presents an approach how to combine them for the case of C functions without loops. The essence of the approach is to build models, both machine code and its specifications in a single logical language, and use target processor ABI to bind machine registers with the parameters of the high-level function. For the successful implementation of this approach, you have to take a number of measures to ensure the compatibility of the high-level specification model with the machine code behavior model. Such measures include the use of a register type in the high-level specifications and the translation of the pre- and postconditions into the abstract predicates. Also in the paper the choice of logical language for building models is made and justified, the most suitable tools for implementing the approach of merging specifications are selected and the evaluation of the system of deductive verification of machine code built on the basis of the proposed approach is made using test examples obtained by compiling C programs without loops.