Custom Instructions Research Articles

Custom RISC-V architecture incorporating memristive in-memory computing

Due to the rise in data-intensive applications, the von Neumann bottleneck is increasingly restricting modern computer architectures, resulting to latency and energy consumption. Addressing this challenge necessitates a CMOS-compatible solution with high energy efficiency and significant parallelism. Utilizing resistive switching components within a 1T1R crossbar array and the application of Stanford RRAM model, this paper suggests an original method for in-memory computing. Moreover, this work shows a new way to advance the popular RISC-V architecture by including memristive crossbar array. It does this by adding a custom instruction set, special hardware blocks, and the Scouting Logic Scheme. These modifications serve both as a comprehensive testbed for the memory system and a proof of concept for the future integration of memristors in computing architectures. The proposed design undergoes extensive testing and power analysis to validate its functionality and performance under various conditions. The results demonstrate significant improvements in computational efficiency and energy savings, highlighting the potential of memristor-based in-memory computing systems to overcome current architectural limitations.

Ditching the Queue: Optimizing Coprocessor Utilization with Out-of-Order CPUs on Compact Systems on Chip

The growing demand for high-performance and energy-efficient processing in edge-oriented Systems-on-Chip is driving the adoption of dedicated integrated circuits that accelerate computationally intensive workloads. To minimize area and performance overhead, low-power, general-purpose CPUs are often tightly coupled with domain-specific coprocessors implementing custom instructions, thereby delivering higher throughput and reduced memory traffic. However, commonly used in-order CPUs are not optimized for instruction-level parallelism, leading to stalls in the instruction stream while waiting for long-latency coprocessor operations and under-utilization of the coprocessor while executing other instructions. This work investigates the benefits of replacing simple in-order cores with a more complex out-of-order architecture to dynamically schedule instructions for the main core and coprocessor, optimizing resource utilization and reducing execution time. To ensure generality, an in-depth analysis was carried out by offloading instructions to a custom dummy coprocessor capable of emulating iterative and pipelined operations with arbitrary latency. Various workloads simulating real-world applications were executed on two variants of an open-source microcontroller equipped with a recent out-of-order core and the state-of-the-art CV32E40X in-order core, respectively. Results from Register Transfer Level simulations show that the former configuration executes up to 60% more instructions per cycle, with a modest 12% system area overhead on a 65 nm CMOS technology node.

Tracking accelerator based on RISC-V custom instructions for GNSS receiver

Tracking accelerator based on RISC-V custom instructions for GNSS receiver

Reinforcement Learning for Selecting Custom Instructions Under Area Constraint

Reinforcement Learning for Selecting Custom Instructions Under Area Constraint

Compact Instruction Set Extensions for Dilithium

Post-quantum cryptography is considered to provide security against both traditional and quantum computer attacks. Dilithium is a digital signature algorithm that derives its security from the challenge of finding short vectors in lattices. It has been selected as one of the standardizations in the NIST post-quantum cryptography project. Hardware-software co-design is a commonly adopted implementation strategy to address various implementation challenges, including limited resources, high performance, and flexibility requirements. In this study, we investigate using compact instruction set extensions (ISEs) for Dilithium, aiming to improve software efficiency with low hardware overheads. To begin with, we propose tightly coupled accelerators that are deeply integrated into the RISC-V processor. These accelerators target the most computationally demanding components in resource-constrained processors, such as polynomial generation, Number Theoretic Transform (NTT), and modular arithmetic. Next, we design a set of custom instructions that seamlessly integrate with the RISC-V base instruction formats, completing the accelerators in a compact manner. Subsequently, we implement our ISEs in a chip design for the Hummingbird E203 core and conduct performance benchmarks for Dilithium utilizing these ISEs. Additionally, we evaluate the resource consumption of the ISEs on FPGA and ASIC technologies. Compared to the reference software implementation on the RISC-V core, our co-design demonstrates a remarkable speedup factor ranging from 6.95 to 9.96. This significant improvement in performance is achieved by incorporating additional hardware resources, specifically, a 35% increase in LUTs, a 14% increase in FFs, 7 additional DSPs, and no additional RAM. Furthermore, compared to the state-of-the-art approach, our work achieves faster speed performance with a reduced circuit cost. Specifically, the usage of additional LUTs, FFs, and RAMs is reduced by 47.53%, 50.43%, and 100%, respectively. On ASIC technology, our approach demonstrates 12, 412 cell counts. Our co-design provides a better tradeoff implementation on speed performance and circuit overheads.

ChatGPT for editors: enhancing efficiency and effectiveness

This tutorial examines how ChatGPT can assist journal editors in improving the efficiency and effectiveness of academic publishing. It highlights ChatGPT’s key characteristics, focusing on the use of “Custom instructions” to generate tailored responses and plugin integration for accessing up-to-date information. The tutorial presents practical advice and illustrative examples to demonstrate how editors can adeptly employ these features to improve their work practices. It covers the intricacies of developing advanced prompts and the application of zero-shot and few-shot prompting techniques across a range of editorial tasks, including literature reviews, training novice reviewers, and improving language quality. Furthermore, the tutorial addresses potential challenges inherent in using ChatGPT, which include a lack of precision and sensitivity to cultural nuances, the presence of biases, and a limited vocabulary in specialized fields, among others. The tutorial concludes by advocating for an integrated approach, combining ChatGPT’s technological advancements with the critical insight of human editors. This approach emphasizes that ChatGPT should be recognized not as a replacement for human judgment and expertise in editorial processes, but as a tool that plays a supportive and complementary role.

RISC-V Custom Instructions of Elementary Functions for IoT Endpoint Devices

RISC-V Custom Instructions of Elementary Functions for IoT Endpoint Devices

A 28-nm 8-bit Floating-Point Tensor Core-Based Programmable CNN Training Processor With Dynamic Structured Sparsity

Training deep/convolutional neural networks (DNNs/CNNs) requires a large amount of memory and iterative computation, which necessitates speedup and energy reduction, especially for edge devices with resource/energy constraints. In this work, we present an 8-bit floating-point (FP8) training the processor which implements: 1) highly parallel tensor cores (fused multiply–add trees) that maintain high utilization throughout forward propagation (FP), backward propagation (BP), and weight update (WU) phases of the training process; 2) hardware-efficient channel gating for dynamic output activation sparsity; 3) dynamic weight sparsity (WS) based on group Lasso; and 4) gradient skipping based on the FP prediction error. We develop a custom instruction set architecture (ISA) to flexibly support different CNN topologies and training parameters. The 28-nm prototype chip demonstrates large improvements in floating-point operations (FLOPs) reduction (7.3 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> ), energy efficiency (16.4 TFLOPS/W), and overall training latency speedup (4.7 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> ), for both supervised and self-supervised training tasks.

PIMCA: A Programmable In-Memory Computing Accelerator for Energy-Efficient DNN Inference

This article presents a programmable in-memory computing accelerator (PIMCA) for low-precision (1–2 b) deep neural network (DNN) inference. The custom 10T1C bitcell in the in-memory computing (IMC) macro has four additional transistors and one capacitor to perform capacitive-coupling-based multiply and accumulation (MAC) in analog-mixed-signal (AMS) domain. A macro containing <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$256\ttimes 128$</tex-math> </inline-formula> bitcells can simultaneously activate all the rows, and as a result, it can perform a matrix-vector multiplication (VMM) in one cycle. PIMCA integrates 108 of such IMC static random-access memory (SRAM) macros with the custom six-stage pipeline and the custom instruction set architecture (ISA) for instruction-level programmability. The results of IMC macros are fed to a single-instruction-multiple-data (SIMD) processor for other computations such as partial sum accumulation, max-pooling, activation functions, etc. To effectively use the IMC and SIMD datapath, we customize the ISA especially by adding hardware loop support, which reduces the program size by up to 73%. The accelerator is prototyped in a 28-nm technology, and integrates a total of 3.4-Mb IMC SRAM and 1.5-Mb off-the-shelf activation SRAM, demonstrating one of the largest IMC accelerators to date. It achieves the system-level energy efficiency of 437 TOPS/W and the peak throughput of 49 TOPS at the 42-MHz clock frequency and 1-V supply for the VGG9 and the ResNet-18 on the CIFAR-10 dataset.

Architecture Support for Bitslicing

The bitsliced programming model has shown to boost the throughput of software programs. However, on a standard architecture, it exerts a high pressure on register access, causing memory spills and restraining the full potential of bitslicing. In this work, we present architecture support for bitslicing in a System-on-Chip. Our hardware extensions are of two types; internal to the processor core, in the form of custom instructions, and external to the processor, in the form of direct memory access module with support for data transposition. We present a comprehensive performance evaluation of the proposed enhancements in the context of several RISC-V ISA definitions (RV32I, RV64I, RV32B, RV64B). The proposed 14 new custom instructions use 1.5× fewer registers compared to the equivalent functionality expressed using RISC-V instructions. The integration of those custom instructions in a 5-stage pipelined RISC-V RV32I core incurs 10.21% and 12.72% overhead respectively in area and cell count using the SkyWater 130 nm standard cell library. The proposed bitslice transposition unit with DMA provides a further speedup, changing the quadratic increase in execution time of data transposition to linear. Finally, we demonstrate a comprehensive performance evaluation using a set of benchmarks of lightweight and masked ciphers.

Design of a nonvolatile-register-embedded RISC-V CPU with software-controlled data-retention and hardware-acceleration functions

This paper describes the design of a nonvolatile CPU based on RISC-V that is an open-source and highly flexible instruction set architecture. This CPU incorporates nonvolatile registers utilizing magnetic tunnel junction (MTJ) device, as well as custom instructions specific to the control of these nonvolatile registers and an accelerator module embedded into the CPU architecture. These techniques enable efficient execution of intermittent operations suitable for energy-limited internet-of-things (IoT) applications. Through performance evaluation of the CPU designed in a 55-nm CMOS/MTJ-hybrid process technology, we show that our CPU can save up to 56.9% of power consumption compared to conventional ones, with an average power consumption of 3.91 μW/MHz.

A Flexible and High-Performance Lattice-Based Post-Quantum Crypto Secure Coprocessor

Progress of quantum computing technology seriously threaten the industrial information security based on traditional public-key cryptosystem. Thus, the cryptosystem with anti-quantum attack characteristics is gradually becoming a significant research in the security field. In this article, a flexible and high-performance secure coprocessor is designed for security in industrial processes, which can execute the post-quantum cryptographic algorithm Saber efficiently. Custom instruction set and arithmetic accelerators are proposed to effectively optimize the flexibility of system architecture, and improve the performance of calculation. The hardware implementation results show that the maximum operating frequency of the coprocessor can reach 345 MHz. Compared with related state-of-the-art works, it achieves the highest operating frequency on the same Xilinx UltraScale+ FPGA platform, performing the encryption and decryption operations within 13.5 and 15.4 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">μ</i> s, respectively. Meanwhile, this article achieves 1.7/3.1/5.9× area-time product improvements in look-up table flip-flop block memory storage with good flexibility.

Efficient Binary Weight Convolutional Network Accelerator for Speech Recognition.

Speech recognition has progressed tremendously in the area of artificial intelligence (AI). However, the performance of the real-time offline Chinese speech recognition neural network accelerator for edge AI needs to be improved. This paper proposes a configurable convolutional neural network accelerator based on a lightweight speech recognition model, which can dramatically reduce hardware resource consumption while guaranteeing an acceptable error rate. For convolutional layers, the weights are binarized to reduce the number of model parameters and improve computational and storage efficiency. A multichannel shared computation (MCSC) architecture is proposed to maximize the reuse of weight and feature map data. The binary weight-sharing processing engine (PE) is designed to avoid limiting the number of multipliers. A custom instruction set is established according to the variable length of voice input to configure parameters for adapting to different network structures. Finally, the ping-pong storage method is used when the feature map is an input. We implemented this accelerator on Xilinx ZYNQ XC7Z035 under the working frequency of 150 MHz. The processing time for 2.24 s and 8 s of speech was 69.8 ms and 189.51 ms, respectively, and the convolution performance reached 35.66 GOPS/W. Compared with other computing platforms, accelerators perform better in terms of energy efficiency, power consumption and hardware resource consumption.

The Design of Optimized RISC Processor for Edge Artificial Intelligence Based on Custom Instruction Set Extension

The Design of Optimized RISC Processor for Edge Artificial Intelligence Based on Custom Instruction Set Extension

A Lightweight Posit Processing Unit for RISC-V Processors in Deep Neural Network Applications

Nowadays, two groundbreaking factors are emerging in neural networks. First, there is the RISC-V open instruction set architecture (ISA) that allows a seamless implementation of custom instruction sets. Second, there are several novel formats for real number arithmetic. In this work, we combined these two key aspects using the very promising posit format, developing a light Posit Processing Unit (PPU-light). We present an extension of the base RISC-V ISA that allows the conversion between 8 or 16-bit posits and 32-bit IEEE Floats or fixed point formats in order to offer a compressed representation of real numbers with little-to-none accuracy degradation. Then we elaborate on the hardware and software toolchain integration of our PPU-light inside the Ariane RISC-V core and its toolchain, showing how little it impacts in terms of circuit complexity and power consumption. Indeed, only 0.36% of the circuit is devoted to the PPU-light while the full RISC-V core occupies the 33% of the overall circuit complexity. Finally we present the impact of our PPU-light on a deep neural network task, reporting speedups up to 10 on sample inference processing time.

AI-PiM—Extending the RISC-V processor with Processing-in-Memory functional units for AI inference at the edge of IoT

The recent advances in Artificial Intelligence (AI) achieving “better-than-human” accuracy in a variety of tasks such as image classification and the game of Go have come at the cost of exponential increase in the size of artificial neural networks. This has lead to AI hardware solutions becoming severely memory-bound and scrambling to keep-up with the ever increasing “von Neumann bottleneck”. Processing-in-Memory (PiM) architectures offer an excellent solution to ease the von Neumann bottleneck by embedding compute capabilities inside the memory and reducing the data traffic between the memory and the processor. But PiM accelerators break the standard von Neumann programming model by fusing memory and compute operations together which impedes their integration in the standard computing stack. There is an urgent requirement for system-level solutions to take full advantage of PiM accelerators for end-to-end acceleration of AI applications. This article presents AI-PiM as a solution to bridge this research gap. AI-PiM proposes a hardware, ISA and software co-design methodology which allows integration of PiM accelerators in the RISC-V processor pipeline as functional execution units. AI-PiM also extends the RISC-V ISA with custom instructions which directly target the PiM functional units resulting in their tight integration with the processor. This tight integration is especially important for edge AI devices which need to process both AI and non-AI tasks on the same hardware due to area, power, size and cost constraints. AI-PiM ISA extensions expose the PiM hardware functionality to software programmers allowing efficient mapping of applications to the PiM hardware. AI-PiM adds support for custom ISA extensions to the complete software stack including compiler, assembler, linker, simulator and profiler to ensure programmability and evaluation with popular AI domain-specific languages and frameworks like TensorFlow, PyTorch, MXNet, Keras etc. AI-PiM improves the performance for vector-matrix multiplication (VMM) kernel by 17.63x and provides a mean speed-up of 2.74x for MLPerf Tiny benchmark compared to RV64IMC RISC-V baseline. AI-PiM also speeds-up MLPerf Tiny benchmark inference cycles by 2.45x (average) compared to state-of-the-art Arm Cortex-A72 processor.

Custom Instructions for Networked Processor Templates

As Field-programmable Gate Arrays (FPGAs) continue to increase in size and capability, there is an increasing need to develop performant designs in good time. Networked processor templates support scalable on-chip parallelism while avoiding the complexity and long run time of FPGA synthesis tools. The performance of the resulting designs can be further enhanced by adding custom instructions to processors in the network. We show how to systematically choose parts of a design to implement as custom instructions. We illustrate the use of performance and area models targeting a particular networked processor template to explore design trade-offs. Various applications, such as N-body simulation and dissipative particle dynamics, demonstrate how hardware acceleration based on custom instructions can target state-of-the-art FPGAs.

NTT Architecture for a Linux-Ready RISC-V Fully-Homomorphic Encryption Accelerator

This paper proposes two architectures for the acceleration of Number Theoretic Transforms (NTTs) using a novel Montgomery-based butterfly. We first design a custom NTT hardware accelerator for Field-Programmable Gate Arrays (FPGAs). The butterfly architecture is expanded to a Modular Arithmetic Logic Unit (MALU) and for greater reuse and easier programmability a six-stage pipeline Linux-ready RISC-V core is extended with custom instructions. The performance of the proposed architectures is assessed on a Xilinx Ultrascale+ FPGA and with an Application-Specific Integrated Circuit (ASIC) on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$28{n}\text{m}$ </tex-math></inline-formula> CMOS technology. In FPGA, the results for custom acceleration show reductions of 30%, 90% and 42% in the number of Lookup tables (LUTs) and registers, Block RAMs (BRAMs) and Digital Signal Processors (DSPs), while providing a speedup of 1.9 times, in comparison with the state of the art. The ASIC results show that at 1 GHz the proposed architecture is in average 45% and 52% less area and power hungry, respectively, compared to the state of the art. Furthermore, the proposed MALU, operating as an additional execution unit, increases the overall area of the extended RISC-V core by only 10%, without significant changes in the frequency of operation.

Audio Denoising Coprocessor Based on RISC-V Custom Instruction Set Extension

As a typical active noise control algorithm, Filtered-x Least Mean Square (FxLMS) is widely used in the field of audio denoising. In this study, an audio denoising coprocessor based on Retrenched Injunction System Computer-V (RISC-V), a custom instruction set extension was designed and a software and hardware co-design was adopted; based on the traditional pure hardware implementation, the accelerator optimization design was carried out, and the accelerator was connected to the RISC-V core in the form of coprocessor. Meanwhile, the corresponding custom instructions were designed, the compiling environment was established, and the library function of coprocessor acceleration instructions was established by embedded inline assembly. Finally, the active noise control (ANC) system was built and tested based on Hbird E203-Core, and the test data were collected through an audio analyzer. The results showed that the audio denoising algorithm can be realized by combining a heterogeneous System on Chip (SoC) with a hardware accelerator, and the denoising effect was approximately 8 dB. The number of instructions consumed by testing custom instructions for specific operations was reduced by approximately 60%, and the operation acceleration effect was significant.

Audio Denoising Coprocessor Based on RISC-V Custom Instruction Set Extension

As a typical active noise control algorithm, FxLMS is widely used in the field of audio denoising. In this paper, an audio denoising coprocessor based on RISC-V custom instruction set extension was designed, and the idea of software and hardware co-design was adopted; based on the traditional pure-hardware implementation, the accelerator optimization design was carried out, and the accelerator was connected to RISC- V core in the form of coprocessor. Meanwhile, the corresponding custom instructions were designed, the compiling environment was established, and the library function of coprocessor acceleration instructions was established by embedded inline assembly. Finally, the ANC system was built and tested based on E203-SoC, and the test data was collected by audio analyzer. The results showed that the audio denoising algorithm could be realized by combining heterogeneous SoC with hardware accelerator, and the denoising effect was about 8dB. The number of instructions consumed by testing custom instructions for specific operations was reduced by about 60%, and the operation acceleration effect was significant.