Code Generation Techniques Research Articles

Handling complexity of large high-level specifications in simulation environment: the case of transformational model-battle management

Modeling system requirements at a high level can significantly facilitate the communication of mental models across highly heterogeneous entities. Two major challenges may arise during such a process. The first is the degree to which details of the higher-level specifications can capture and effectively handle. Especially, when the model contains more nuanced descriptions of parallel execution and temporal pacing. The second challenge arises when realizing such specifications downstream due to ambiguities and incompleteness encountered in higher-level semiformal artifacts. We propose a formal specification and transformation of higher-level artifacts, particularly in the operational activity model of the Department of Defense Architecture Framework (OV-5 DoDAF). We create an isomorphic activity model corresponding to each activity in the Transformational Model-Battle Management. Then, we generate a code for simulating the models as Markov Discrete Event System Specifications. We demonstrate the size and other aspects of the experiments that can be conducted based on the generated simulation and the level of complexity in such an approach. We highlight aspects of managing complexity using model transformations and code generation techniques in a full-cycle model-based system engineering approach, expediting the process toward arriving at the concrete realization of simulation models that only originated from minimal high-level metamodels and artifacts.

A Framework for Improving Portability and Ensuring Correctness of Operating System Kernels

Traditional embedded Real-Time Operating Systems (RTOS) or Basic Software (BSW) implementations typically require manual porting to new hardware platforms. However, this approach can be time-consuming and error-prone, especially given the frequent introduction of new or upgraded hardware architectures. In addition, traditional testing methods may not fully capture the complexity and nuances of the system, making it difficult to ensure correctness and dependability. To address these challenges, we propose a comprehensive methodology that integrates formal methods, a WCET tool, and a code generation technique. We use formal methods to create models and verify their correctness against functional and non-functional specifications or properties such as safety, liveness, and timing. We use the WCET tool as a microarchitecture analyzer to analyze the low-level binary code. The intermediate results of the tool are used to verify the correctness of the software implementation against the runtime effects of the hardware, such as data/memory race conditions. Finally, the code is generated from the formal models. Our proposed framework simplifies the maintenance of RTOS or BSW implementations while ensuring their correctness and partially automating portability to new hardware architectures.

Research on Code Generation Technology based on LLM Pre-training

In recent years, the continuous improvement and rapid development of large language model (LLM) technology, and the pre-trained code generation technology involved in it has attracted extensive attention in the industry. Through LLM, the conversion from the well-known natural language (NL) to the programming language (PL) written by professional code practitioners can be realized, which greatly reduces the threshold of programming language, and has demonstrated significant performance and advantages in code generation tasks through pre-training. This paper systematically sorts out, researches and summarizes the pre-trained code generation techniques in recent years. Firstly, the development time roadmap of the pre-trained model related to code generation is extracted from the relevant research results. Secondly, the characteristics of different code generation pre-trained models are sorted out and summarized. At the same time, the evaluation mechanism and dataset for different pre-trained code generation models are given, and the research data are compared and analyzed. Finally, combined with the current development situation, the future development direction of code generation technology is prospected.

Structured Chain-of-Thought Prompting for Code Generation

Large Language Models (LLMs) have shown impressive abilities in code generation. Chain-of-Thought (CoT) prompting is the state-of-the-art approach to utilizing LLMs. CoT prompting asks LLMs first to generate CoTs ( i.e., intermediate natural language reasoning steps) and then output the code. However, the accuracy of CoT prompting still can not satisfy practical applications. For example, gpt-3.5-turbo with CoT prompting only achieves 53.29% Pass@1 in HumanEval. In this paper, we propose Structured CoTs (SCoTs) and present a novel prompting technique for code generation named SCoT prompting. Our motivation is that human developers follow structured programming. Developers use three programming structures ( i.e., sequential, branch, and loop) to design and implement structured programs. Thus, we ask LLMs to use three programming structures to generate SCoTs (structured reasoning steps) before outputting the final code. Compared to CoT prompting, SCoT prompting explicitly introduces programming structures and unlocks the structured programming thinking of LLMs. We apply SCoT prompting to two LLMs ( i.e., gpt-4-turbo, gpt-3.5-turbo, and DeepSeek Coder-Instruct-{1.3B, 6.7B, 33B}) and evaluate it on three benchmarks ( i.e., HumanEval, MBPP, and MBCPP). SCoT prompting outperforms CoT prompting by up to 13.79% in Pass@1. SCoT prompting is robust to examples and achieves substantial improvements. The human evaluation also shows human developers prefer programs from SCoT prompting.

Predictable and optimized single-path code for predicated processors

Single-path code is a code generation technique for real-time systems that reduces execution time variability. However, doing so can incur significant execution-time overhead and does not guarantee constant execution times. In this paper, we address the performance challenges of single-path code and solve the variability issue. We present the repetition dominance relation to identify and optimize code blocks that are always executed a fixed number of times. We show that single-path code’s instructions are uniquely easy to schedule, and we explore an extension to the Patmos architecture that allows additional instruction types in the second issue slot. Lastly, we present two techniques for ensuring that functions always perform the same number of accesses to memory, resulting in programs with constant execution time. We compare the performance of single-path code to that of statically analyzed traditional code. Our results show that single-path code’s performance is mostly competitive while outright superior in several cases. However, pathological cases of poor performance are still observed.

Two Birds with One Stone: Boosting Code Generation and Code Search via a Generative Adversarial Network

Automatically transforming developers' natural language descriptions into source code has been a longstanding goal in software engineering research. Two types of approaches have been proposed in the literature to achieve this: code generation, which involves generating a new code snippet, and code search, which involves reusing existing code. However, despite existing efforts, the effectiveness of the state-of-the-art techniques remains limited. To seek for further advancement, our insight is that code generation and code search can help overcome the limitation of each other: the code generator can benefit from feedback on the quality of its generated code, which can be provided by the code searcher, while the code searcher can benefit from the additional training data augmented by the code generator to better understand code semantics. Drawing on this insight, we propose a novel approach that combines code generation and code search techniques using a generative adversarial network (GAN), enabling mutual improvement through the adversarial training. Specifically, we treat code generation and code search as the generator and discriminator in the GAN framework, respectively, and incorporate several customized designs for our tasks. We evaluate our approach in eight different settings, and consistently observe significant performance improvements for both code generation and code search. For instance, when using NatGen, a state-of-the-art code generator, as the generator and GraphCodeBERT, a state-of-the-art code searcher, as the discriminator, we achieve a 32% increase in CodeBLEU score for code generation, and a 12% increase in mean reciprocal rank for code search on a large-scale Python dataset, compared to their original performances.

SPECIFICATION FORMALIZATION OF STATE CHARTS FOR COMPLEX SYSTEM MANAGEMENT

This article presents a formalization approach for the requirements of object-oriented programs with state machines, using a spacecraft control system as a case study. It proposes a state pattern implementation, where each state is represented as a class with clearly defined responsibilities, and the transitions between states are controlled by the state objects themselves. Additionally, the application of model checking, theorem proving, and code generation techniques are discussed. The effectiveness of the proposed approach in ensuring compliance with the specified requirements is demonstrated, while also identifying potential drawbacks and limitations of the approach. The implementation is validated using a range of formal verification techniques, including model checking and theorem proving. The article also discusses how the approach can be extended and applied to other complex systems. Overall, the valuable insights into the formalization of requirements for object-oriented programs with state machines are provided, offering a practical and effective approach for verifying the correctness and completeness of such implementations. The results of this work have important implications for the development of safety-critical systems and can potentially improve the quality and reliability of software systems in various domains. By using mathematical models and rigorous formal methods, it is possible to detect and eliminate errors early in the development process, leading to higher confidence in the correctness of the final product. Future research in this area could explore the use of more advanced techniques, such as model-driven development and automatic code synthesis, to further streamline the software development process. Additionally, the development of more efficient and user-friendly tools could make these techniques more accessible to a wider range of developers and organizations. Altogether, the combination of formal methods and software engineering has the potential to revolutionize the way software systems are designed, developed, and verified, leading to safer and more reliable software for critical applications.

XFC: Enabling automatic and fast operator synthesis for mobile deep learning compilation

Deploying a deep learning model relies on highly optimized implementations for all tensor operators in the model, especially on mobile devices with limited hardware resources. To relieve the burden of manual optimization, researchers propose deep learning compilers to automatically optimize operators with auto-tuning and code generation techniques. The auto-tuning system typically constructs a large tuning space, from which various tensor programs are sampled and evaluated to find the best implementation. Unfortunately, this process is quite time-consuming, often requiring hours for a single operator. To address this issue, this paper presents XFC, a framework that enables fast performance tuning by operator synthesis. The key idea of XFC is to abstract the hand-tuning process and generate tensor programs by bottom-up and hierarchical construction. We implemented XFC based on TVM and conduct extensive experiments to verify its effectiveness. Experiments show that, for various operators and mobile devices, XFC significantly reduces the tuning time from hours to seconds (over 700× speedup) while ensuring comparable execution performance for compiled operators (11.5% performance loss on average).

A Survey of Healthcare Professionals Attitude and Awareness towards Quick Response Codes and Review of its Possible Applications in Medical Education and Pharmaceutical Industry

Medicine is rapidly adopting newer technologies such as Quick Response (QR) codes to effectively and efficiently manage, share and store information. QR codes can have a wide variety of applications in the healthcare, medical education and pharmaceutical industry. This article aims to explore the applications of Quick Response code in Medicine and other affiliated sectors. Through this survey we have tried to score the attitude and awareness of healthcare professionals towards the viability of QR codes. The study sample was of n = 142 respondents. Majority of respondents were aware about the utility of QR codes in day-today aspects and academics with Online transactions being the predominant usage. 84.5% respondents were aware of the technology involved in QR codes and 83.1% respondents had used the QR code in different scenarios. Despite the awareness only 21.1% respondents knew how to create a QR code. Overwhelming majority of 80.3% respondents replied that they would prefer access to educational material and recording of attendance through QR codes. According to the findings, awareness of technology and use of QR codes is not a significant factor in QR code acceptance in accessing study material and attendance. But we have a significant p-value (p value = 0.017) when acceptance of QR codes in medical education was significant among healthcare professionals who were exposed to QR codes in the Department of Pharmacology. Our findings further suggest that most healthcare professionals have convenient access to the technology essential to generate QR codes, and yet, about 21.1% of healthcare professionals are unaware of the technique for QR code generation. Based on this finding, we conclude that knowledge of QR codes and use of QR codes outside of the medical setup doesn’t affect the acceptability of QR codes in healthcare professionals, but when they’re exposed to QR codes in the medical setup, there is a significant correlation in acceptance of QR codes for study materials and attendance. From our results, we can conclude that most healthcare workers and medical students approve of more integration of QR codes in medical workflow and agree with the convenience and ease of use and easier access to information. Keywords: Medical education; Behaviour modification; Pharmaceutical Industry; Healthcare; Quick Response code

Application of the Universal Quench Detection System to the Protection of the High-Luminosity LHC Magnets at CERN

The Universal Quench Detection System (UQDS) has been primarily developed to detect quenches in various superconducting magnets of LHC’s High Luminosity upgrade (HL-LHC). The functionality of the system, which comprises insulated high-resolution digitizing front-end channels and a central processing element, is mainly defined by the configuration of the central FPGA (Field Programmable Gate Array). Additional elements such as redundant power supplies, configurable interlocking capabilities and a communications controller are completing the functionality of the system. The system’s architecture is designed to be flexible enough to detect quenches in all superconducting elements of HL-LHC. The application-specific digital signal processing and quench detection algorithms are implemented in the firmware of the FPGA and thus can be changed according to the required specifications. To facilitate this process, the firmware has been structured accordingly and automated code generation techniques are used. The strategy of testing prototypes of the quench detection system with prototypes of the magnets allows an early evaluation of the system, minimizing operational issues in the final installation. This strategy has been already applied at CERN to the 11 T dipole magnets in previous years and extended as well to other magnet families of the HL-LHC Project. We give a system description and focus on the specific configurations for three different magnet families. Prototypes of these magnets have been recently tested at CERN. For each of these, specific features and detection methods have been developed, implemented and evaluated during the magnet tests.

Reimplementing the Wheel: Teaching Compilers with a Small Self-Contained One

We report on a one-semester compiler construction course based on the idea of implementing a small self-contained compiler for a small model language from scratch, not using other compiler construction frameworks. The course is built around an evolving family of languages with increasing expressiveness and complexity, which finally is crowned by a language with first-class functions, S-expressions, pattern matching, and garbage collection. The code generation technique is based on the idea of symbolic interpreters, which allows to implement a robust albeit not a very efficient native code generator. We give the motivation for the course, describe its structure, and report some results of teaching based on students' post-course surveys.

Towards automatic finite-element methods for geodynamics via Firedrake

Abstract. Firedrake is an automated system for solving partial differential equations using the finite-element method. By applying sophisticated performance optimisations through automatic code-generation techniques, it provides a means of creating accurate, efficient, flexible, easily extensible, scalable, transparent and reproducible research software that is ideally suited to simulating a wide range of problems in geophysical fluid dynamics. Here, we demonstrate the applicability of Firedrake for geodynamical simulation, with a focus on mantle dynamics. The accuracy and efficiency of the approach are confirmed via comparisons against a suite of analytical and benchmark cases of systematically increasing complexity, whilst parallel scalability is demonstrated up to 12 288 compute cores, where the problem size and the number of processing cores are simultaneously increased. In addition, Firedrake's flexibility is highlighted via straightforward application to different physical (e.g. complex non-linear rheologies, compressibility) and geometrical (2-D and 3-D Cartesian and spherical domains) scenarios. Finally, a representative simulation of global mantle convection is examined, which incorporates 230 Myr of plate motion history as a kinematic surface boundary condition, confirming Firedrake's suitability for addressing research problems at the frontiers of global mantle dynamics research.

Application of the automatic code generation and symbolic computation techniques to numerical solution of PDE problems

A software package for the numerical solution of systems of linear and nonlinear partial differential equations by grid methods is presented, which combines symbolic computations and automatic code generation techniques. Discretization of differential equations is carried out by the methods of finite volumes and finite differences on two-dimensional and three-dimensional regular rectangular grids. The solution of the systems of nonlinear algebraic equations obtained as a result of discretization is performed by Newton’s method with employment of third-party SLAE solvers. The generated code can exploit shared and distributed memory parallelisms and take advantage of heterogeneous computations. The results of the numerical solutions of two sample CFD problems of the fluid flow generation in a cylindrical cavity filled with electrically conductive liquid driven the by traveling and rotating magnetic fields are presented.

TAM: A Front-End to an Auto-Parallelizing Compiler

The multi-core architecture has revolutionized the parallel computing. Despite this, the modern age compilers have a long way to achieve auto-parallelization. Through this paper, we introduce a language that encouraging the auto-parallelization. We are also introducing Front-End for our auto-parallelizing compiler. Later, we examined our compiler employing a different number of core and verify results based on different metrics based on total compilation time, memory utilization, power utilization and CPU utilization. At last, we learned that parallelizing multiple files engage more CPU resources, memory and energy, but it finishes the task at hand in less time. In this paper, we have proposed a loop code generation technique that makes the generation of nested loop IR code faster by dividing the blocks into some extra code blocks using a modular approach. Our TAM compiler technique speedup by 7.506, 5.283 and 2.509 against sequential compilation when we utilized 8, 4 and 2 cores respectively. We observed that the CPU utilization of the TAM compiler reaches the maximum permissible limit when an optimal parallelizable instance is compiled.

EDEN: A High-Performance, General-Purpose, NeuroML-Based Neural Simulator.

Modern neuroscience employs in silico experimentation on ever-increasing and more detailed neural networks. The high modeling detail goes hand in hand with the need for high model reproducibility, reusability and transparency. Besides, the size of the models and the long timescales under study mandate the use of a simulation system with high computational performance, so as to provide an acceptable time to result. In this work, we present EDEN (Extensible Dynamics Engine for Networks), a new general-purpose, NeuroML-based neural simulator that achieves both high model flexibility and high computational performance, through an innovative model-analysis and code-generation technique. The simulator runs NeuroML-v2 models directly, eliminating the need for users to learn yet another simulator-specific, model-specification language. EDEN's functional correctness and computational performance were assessed through NeuroML models available on the NeuroML-DB and Open Source Brain model repositories. In qualitative experiments, the results produced by EDEN were verified against the established NEURON simulator, for a wide range of models. At the same time, computational-performance benchmarks reveal that EDEN runs from one to nearly two orders-of-magnitude faster than NEURON on a typical desktop computer, and does so without additional effort from the user. Finally, and without added user effort, EDEN has been built from scratch to scale seamlessly over multiple CPUs and across computer clusters, when available.

Flow-Based Programming for Machine Learning

Machine Learning (ML) has gained prominence and has tremendous applications in fields like medicine, biology, geography and astrophysics, to name a few. Arguably, in such areas, it is used by domain experts, who are not necessarily skilled-programmers. Thus, it presents a steep learning curve for such domain experts in programming ML applications. To overcome this and foster widespread adoption of ML techniques, we propose to equip them with domain-specific graphical tools. Such tools, based on the principles of flow-based programming paradigm, would support the graphical composition of ML applications at a higher level of abstraction and auto-generation of target code. Accordingly, (i) we have modelled ML algorithms as composable components; (ii) described an approach to parse a flow created by connecting several such composable components and use an API-based code generation technique to generate the ML application. To demonstrate the feasibility of our conceptual approach, we have modelled the APIs of Apache Spark ML as composable components and validated it in three use-cases. The use-cases are designed to capture the ease of program specification at a higher abstraction level, easy parametrisation of ML APIs, auto-generation of the ML application and auto-validation of the generated model for better prediction accuracy.

Achieving the Performance of All-Bank In-DRAM PIM With Standard Memory Interface: Memory-Computation Decoupling

Processing-in-Memory (PIM) has been actively studied to overcome the memory bottleneck by placing computing units near or in memory, especially for efficiently processing low locality data-intensive applications. We can categorize the in-DRAM PIMs depending on how many banks perform the PIM computation by one DRAM command: per-bank and all-bank. The per-bank PIM operates only one bank, delivering low performance but preserving the standard DRAM interface and servicing non-PIM requests during PIM execution. The all-bank PIM operates all banks, achieving high performance but accompanying design issues like thermal and power consumption. We introduce the memory-computation decoupling execution to achieve the ideal all-bank PIM performance while preserving the standard JEDEC DRAM interface, i.e., performing the per-bank execution, thus easily adapted to commercial platforms. We divide the PIM execution into two phases: memory and computation phases. At the memory phase, we read the bank-private operands from a bank and store them in PIM engines’ registers bank-by-bank. At the computation phase, we decouple the PIM engine from the memory array and broadcast a bank-shared operand using a standard read/write command to make all banks perform the computation simultaneously, thus reaching the computing throughput of the all-bank PIM. For extending the computation phase, i.e., maximizing all-bank execution opportunity, we introduce a compiler analysis and code generation technique to identify the bank-private and the bank-shared operands. We compared the performance of Level-2/3 BLAS, multi-batch LSTM-based Seq2Seq model, and BERT on our decoupled PIM with commercial computing platforms. In Level-3 BLAS, we achieved speedups of 75.8x, 1.2x, and 4.7x compared to CPU, GPU, and the per-bank PIM and up to 91.4% of the ideal all-bank PIM performance. Furthermore, our decoupled PIM consumed less energy than GPU and the per-bank PIM by 72.0% and 78.4%, but 7.4%, a little more than the ideal all-bank PIM.

A New Structure of 8-Bit 60 MS/s SAR-ADC Using a Reduced Switching Capacitor-DAC Array

This paper presents an analysis of the Reduced Switching Capacitor Digital-to-Analog Converter (RSC-DAC)-based low power Successive Approximation Register Analog to Digital Converter (SAR-ADC). The proposed structure involves the Low voltage Static D-Latch Comparator (LSD-LC) with pre-amplifier operators in two modes (Normal and Hold), the RSC-DAC switching energy, reduced by 93% contrast to the standard Charge Redistribution Switching Capacitor DAC (CRSC-DAC) method, and the Successive Approximation Register (SAR) control logic. The LSD-LC with pre-amplifier consists of a latch circuit and a pre-amplifier. The pre-amplifier is often used to eliminate the DC offset voltage and kickback noise without substantially weakening the Signal-to-Noise Ratio (SNR) to drive the main circuit while the latch is needed for comparison. The linearity parameters such as Integral Nonlinearity, Differential Nonlinearity and effect of parasitic capacitances of the RSC-DAC are analyzed and improved by the new approach named as Adaptive Random Code Generation (ARCG) Technique. The above overall design is implemented in 250-nm CMOS design of the TANNER-EDA tool, consuming 1.74-mW power at 60[Formula: see text]MS/s. The proposed structure has an INL and a DNL, respectively, of +0.18/[Formula: see text] LSB and +0.11/[Formula: see text]0.05 LSB.

Compiling for time-predictability with dual-issue single-path code

Designed for real-time systems, the Patmos instruction-set architecture’s features ensure a high degree of predictability. One such feature is its dual-issue pipeline, which can issue and execute bundles of up to two instructions at a time. Executing instructions in the second issue slot is a predictable way to increase the throughput of a processor, but without dedicated support from the compiler, this benefit cannot be unlocked. A compiler generates highly predictable programs by generating single-path code. This technique produces code that always follows the same trace of instructions. While Patmos’ compiler can already produce single-path code, it does not assign any instructions to the second issue-slot. This limitation is unfortunate, as single-path code inherently possesses a high degree of instruction-level parallelism. In this paper, we present a single-path code generation technique with support for dual-issue pipelines. It can also support different bundling algorithms, which allows changing algorithms without having to edit other parts of the compiler. We present a simple bundling algorithm plugged into the single-path code generator. It looks for branches and bundles the basic blocks on each path of the branch. While this specific bundling algorithm is too simple to provide a real-world benefit, it highlights the potential that further work on bundling algorithms can unlock.

RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices

Mobile devices are becoming an important carrier for deep learning tasks, as they are being equipped with powerful, high-end mobile CPUs and GPUs. However, it is still a challenging task to execute 3D Convolutional Neural Networks (CNNs) targeting for real-time performance, besides high inference accuracy. The reason is more complex model structure and higher model dimensionality overwhelm the available computation/storage resources on mobile devices. A natural way may be turning to deep learning weight pruning techniques. However, the direct generalization of existing 2D CNN weight pruning methods to 3D CNNs is not ideal for fully exploiting mobile parallelism while achieving high inference accuracy. This paper proposes RT3D, a model compression and mobile acceleration framework for 3D CNNs, seamlessly integrating neural network weight pruning and compiler code generation techniques. We propose and investigate two structured sparsity schemes i.e., the vanilla structured sparsity and kernel group structured (KGS) sparsity that are mobile acceleration friendly. The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism. We propose a reweighted regularization pruning algorithm to achieve the proposed sparsity schemes. The inference time speedup due to sparsity is approaching the pruning rate of the whole model FLOPs (floating point operations). RT3D demonstrates up to 29.1x speedup in end-to-end inference time comparing with current mobile frameworks supporting 3D CNNs, with moderate 1%~1.5% accuracy loss. The end-to-end inference time for 16 video frames could be within 150 ms, when executing representative C3D and R(2+1)D models on a cellphone. For the first time, real-time execution of 3D CNNs is achieved on off-the-shelf mobiles.