Hardware-software Co-design Research Articles

A Practical Multiobjective Learning Framework for Optimal Hardware-Software Co-Design of Control-on-a-Chip Systems

A Practical Multiobjective Learning Framework for Optimal Hardware-Software Co-Design of Control-on-a-Chip Systems

TaPaFuzz: Hardware-accelerated RISC-V bare-metal firmware fuzzing using rapid job launches

Fuzz testing serves as a key technique in software security aimed at identifying unexpected program behaviors by repeatedly executing the target program with auto-generated random inputs. Testing is integral to IoT device security but is hampered by the minimal observability features of typical in-market IoT devices. Moreover, the slow nature of a RISC-V software emulation on x86 host CPUs and the inaccuracies introduced by compiling IoT applications to a different ISA for execution on host systems pose significant challenges. Our software-hardware co-design surmounts these hurdles. Fuzzing jobs are prepared and evaluated on a host computer, while the actual execution with high-throughput tracing is performed on an FPGA. Advances in the host-to-FPGA interface together with an accelerated reset procedure between Fuzzer jobs effectively hide the costly host-FPGA communication, increasing the single-thread fuzzing performance by up to factor 11.7x that of the leading QEMU-based fuzzer AFL++ running on a very fast x86 CPU. We demonstrate practical usability by evaluating our framework on a collection of bare-metal applications.

Real-time Blood Pressure Prediction on Wearable Devices using Edge Based Deep Neural Networks: A Hardware-software Co-design Approach

This paper presents the hardware realization of a real-time blood pressure (BP) prediction model for wearable devices, utilizing long short-term memory (LSTM) deep neural networks (DNNs). The proposed system uses both electrocardiogram (ECG) and photoplethysmogram (PPG) signal values for BP prediction. It aims to address the limitations of traditional BP measurement methods, providing a low error, minimal computational overhead, more accurate and convenient alternative system for individuals with hypertension or at risk for cardiovascular diseases. The utilization of split matrix approach leads to a reduction in hardware complexity across the entire system. This technique involves breaking down the larger weight matrices used in the computations of DNNs into smaller matrices. This fragmentation results in a decrease in the complexity of the hardware responsible for performing matrix vector multiplications (MVMs) within LSTMs. The resultant architecture of the predictive model gains several advantages, including a lowered level of complexity in terms of the space occupied by individual cells, decreased processing delay, and reduced power consumption. Furthermore, this approach enables the achievement of a notably improved minimum achievable clock period of 2.972 ns. This prediction model can operate locally on wearable devices, reducing the reliance on cloud computing and improving privacy and security. The performance evaluations are carried out using both analytical and implementation results. The results indicate that the proposed model can be practically applied to real-world problems and can potentially enhance the accuracy of various machine-learning tasks.

Adapting magnetoresistive memory devices for accurate and on-chip-training-free in-memory computing.

Memristors have emerged as promising devices for enabling efficient multiply-accumulate (MAC) operations in crossbar arrays, crucial for analog in-memory computing (AiMC). However, variations in memristors and associated circuits can affect the accuracy of analog computing. Typically, this is mitigated by on-chip training, which is challenging for memristors with limited endurance. We present a hardware-software codesign using magnetic tunnel junction (MTJ)-based AiMC off-chip calibration that achieves software accuracy without costly on-chip training. Hardware-wise, MTJ devices exhibit ultralow cycle-to-cycle variations, as experimentally evaluated over 1 million mass-produced devices. Software-wise, leveraging this, we propose an off-chip training method to adjust deep neural network parameters, achieving accurate AiMC inference. We validate this approach with MAC operations, showing improved transfer curve linearity and reduced errors. By emulating large-scale neural network models, our codesigned MTJ-based AiMC closely matches software baseline accuracy and outperforms existing off-chip training methods, highlighting MTJ's potential in AI tasks.

Automatic Architecture Design for Distributed Quantum Computing

Abstract In distributed quantum computing (DQC), quantum hardware design mainly focuses on providing as many as possible high-quality inter-chip connections. Meanwhile, quantum software tries best to reduce the required number of remote quantum gates between chips. However, this "hardware first, software follows" methodology may not fully exploit the potential of DQC. Inspired by classical software-hardware co-design, this paper explores the design space of application-specific DQC architectures. More specifically, we propose AutoArch, an automated quantum chip network (QCN) structure design tool. With qubits grouping followed by a customized QCN design, AutoArch can generate a near-optimal DQC architecture suitable for target quantum algorithms. Experimental results show that the DQC architecture generated by AutoArch can outperform other general QCN architectures when executing target quantum algorithms.

Hardware–Software Co-Design Enabling Static and Dynamic Sparse Attention Mechanisms

Hardware–Software Co-Design Enabling Static and Dynamic Sparse Attention Mechanisms

Mentor: A Memory-Efficient Sparse-dense Matrix Multiplication Accelerator Based on Column-Wise Product

Sparse-dense matrix multiplication (SpMM) is the performance bottleneck of many high-performance and deep-learning applications, making it attractive to design specialized SpMM hardware accelerators. Unfortunately, existing hardware solutions do not take full advantage of data reuse opportunities of the input and output matrices or suffer from irregular memory access patterns. Their strategies increase the off-chip memory traffic and bandwidth pressure, leaving much room for improvement. We present Mentor , a new approach to designing SpMM accelerators. Our key insight is that column-wise dataflow, while rarely exploited in prior works, can address these issues in SpMM computations. Mentor is a software-hardware co-design approach for leveraging column-wise dataflow to improve data reuse and regular memory accesses of SpMM. On the software level, Mentor incorporates a novel streaming construction scheme to preprocess the input matrix for enabling a streaming access pattern. On the hardware level, it employs a fully pipelined design to unlock the potential of column-wise dataflow further. The design of Mentor is underpinned by a carefully designed analytical model to find the trade-off between performance and hardware resources. We have implemented an FPGA prototype of Mentor . Experimental results show that Mentor achieves speedup by geomean 2.05 × (up to 3.98 ×), reduces the memory traffic by geomean 2.92 × (up to 4.93 ×), and improves bandwidth utilization by geomean 1.38 × (up to 2.89 ×), compared to the state-of-art hardware solutions.

Semantic memory-based dynamic neural network using memristive ternary CIM and CAM for 2D and 3D vision.

The brain is dynamic, associative, and efficient. It reconfigures by associating the inputs with past experiences, with fused memory and processing. In contrast, AI models are static, unable to associate inputs with past experiences, and run on digital computers with physically separated memory and processing. We propose a hardware-software co-design, a semantic memory-based dynamic neural network using a memristor. The network associates incoming data with the past experience stored as semantic vectors. The network and the semantic memory are physically implemented on noise-robust ternary memristor-based computing-in-memory (CIM) and content-addressable memory (CAM) circuits, respectively. We validate our co-designs, using a 40-nm memristor macro, on ResNet and PointNet++ for classifying images and three-dimensional points from the MNIST and ModelNet datasets, which achieves not only accuracy on par with software but also a 48.1 and 15.9% reduction in computational budget. Moreover, it delivers a 77.6 and 93.3% reduction in energy consumption.

Optimized Hardware-Software Co-Design for Kyber and Dilithium on RISC-V SoC FPGA

Optimized Hardware-Software Co-Design for Kyber and Dilithium on RISC-V SoC FPGA

Platform tailored codesign of gate-based quantum simulation

The utility of near-term quantum computers and simulators is likely to rely upon software-hardware codesign, with error-aware algorithms and protocols optimized for the platforms they are run on. Here, we show how knowledge of noise in a system can be exploited to improve the design of gate-based quantum simulation algorithms. We demonstrate this codesign in the context of a trapped ion quantum simulation of the dynamics of a Heisenberg spin model. Specifically, we derive a theoretical noise model describing unitary gate errors due to heating of the ions' collective motion, finding that the temporal correlations in the noise induce an optimal gate depth. We then illustrate how tailored feedforward control, best applied at this optimum, can be used to partially mitigate unitary gate errors and improve the simulation outcome. Our results provide a practical guide to the codesign of gate-based quantum simulation algorithms. Published by the American Physical Society 2024

Codesign of reactor-oriented hardware and software for cyber-physical systems

Modern cyber-physical systems often make use of heterogeneous systems-on-chip with reconfigurable logic to provide adequate computing power and flexible I/O. However, modeling, verifying, and implementing the computations spanning CPUs and reconfigurable logic is still challenging. The hardware and software components are often designed by different teams and at different levels of abstraction, making it hard to reason about the resulting computation. We propose to lift both hardware and software design to the same level of abstraction by using the Lingua Franca coordination language. Lingua Franca is based on a sparse synchronous model that allows modeling concurrency and timing while keeping a sequential model for the actual computation. We define hardware reactors as a subset of the reactor model of computation underlying Lingua Franca. We also present and evaluate reactor-chisel, a hardware runtime implementing the semantics of hardware reactors, and an extension to the Lingua Franca compiler enabling reactor-oriented hardware-software codesign.

Hardware software co-design for leveraging STDP in a memristive neuroprocessor

In neuromorphic computing, different learning mechanisms are being widely adopted to improve the performance of a specific application. Among these techniques, spike-timing-dependent plasticity (STDP) stands out as one of the most favored. STDP is simply managed by the temporal information of an event, which is biologically inspired. However, most of the prior works on STDP are focused on circuit implementation or software simulation for performance evaluation. Previous works also lack a comparative analysis of the performances of different STDP implementations. This study aims to provide a comprehensive assessment of STDP, centering on the performance across various applications such as classification (static and temporal datasets), control, and reservoir computing. Different applications necessitate distinct STDP configurations to achieve optimal performance with the neuroprocessor. Additionally, this work introduces an application-specific integrated circuit design of STDP circuitry. The design is based on current-controlled memristive synapse principles and utilizes 65 nm CMOS technology from IBM. The detailed presentation includes circuitry specifics, layout, and performance parameters such as energy consumption and design area.

Zynq FPGA-Based Acceleration of Kernelized Correlation Filters via High-Level Synthesis of a Custom DFT Block

This study presents a hardware-software co-design implementation of an accelerator for the Kernelized Correlation Filter (KCF) tracking algorithm. Leveraging the High-level synthesis (HLS) and the Zynq heterogeneous platform, the KCF algorithm’s performance is enhanced by using a custom hardware implementation for the computationally intensive Discrete Fourier Transform (DFT) operation. Within this framework, a custom combined DFT and inverse DFT IP, named CDFT, is developed and optimized on the Programmable Logic (PL) side of the Xilinx ZCU102 FPGA, whereas the rest of the KCF algorithm is run with customized Petalinux build on the (Processing System) side. To assess real-world performance, a driver for the CDFT IP and a user application were created to measure metrics like Center Location Error (CLE), Intersection over Union (IoU), and Frame per Second (FPS). The designed DFT accelerator achieves a remarkable speedup of 21x compared to a software DFT implementation. At the algorithm level, the KCF accelerator obtains a 6x speed up with negligible precision loss. In comparison to prior studies employing exclusively hardware implementations, the proposed approach demonstrates a high accuracy at a moderate speed, while there exists potential for further optimizations to enhance its performance even further.

Co-design based FPGA implementation of an efficient new speech hyperchaotic cryptosystem in the transform domain

In this paper a new encryption system has been designed and implemented for real-time speech transmission to reduce bandwidth requirements, increase security and minimize residual intelligibility. To guarantee robustness and lightweight computation, the developed cryptosystem has been carried out in the wavelet transform domain based on a hyperchaotic model to generate mask and permutation keys. The cryptographic system has been designed using a hardware-software (HW/SW) co-design approach by developing several IP-cores in a relatively short development time. The performances and security evaluation of the system have been validated through simulation results followed by an experimental validation through the implementation of an encrypted speech signal transmission between two low cost Nexys-4 DDR FPGA platforms, operating in real-time for both wired and wireless communications. Compared to similar works, high performances have been obtained in terms of bandwidth efficiency due to the use of DWT, limited area of FPGA resources, low power consumption and high security level with a large keyspace that is sufficient to resist against brute force attacks. The designed system can be a very useful solution for many real-time secure integrated voice communication systems, multiple communication purposes, military, professional or personal high level of conversations security.

Fully Binarized Graph Convolutional Network Accelerator Based on In‐Memory Computing with Resistive Random‐Access Memory

Artificial intelligence for graph‐structured data has achieved remarkable success in applications such as recommendation systems, social networks, drug discovery, and circuit annotation. Graph convolutional networks (GCNs) are an effective way to learn representations of various graphs. The increasing size and complexity of graphs call for in‐memory computing (IMC) accelerators for GCN to alleviate massive data transmission between off‐chip memory and processing units. However, GCN implementation with IMC is challenging because of the large memory consumption, irregular memory access, and device nonidealities. Herein, a fully binarized GCN (BGCN) accelerator based on computational resistive random‐access memory (RRAM) through software–hardware codesign is presented. The essential operations including aggregation and combination in GCN are implemented on the RRAM crossbar arrays with cooperation between multiply‐and‐accumulation and content‐addressable memory operations. By leveraging the model quantization and IMC on the RRAM, the BGCN accelerator demonstrates less RRAM usage, high robustness to the device variations, high energy efficiency, and comparable classification accuracy compared to the current state‐of‐the‐art GCN accelerators on both graph classification task using the MUTAG and PTC datasets and node classification task using the Cora and CiteSeer datasets. These results provide a promising approach for edge intelligent systems to efficiently process graph‐structured data.

Compact Instruction Set Extensions for Dilithium

Post-quantum cryptography is considered to provide security against both traditional and quantum computer attacks. Dilithium is a digital signature algorithm that derives its security from the challenge of finding short vectors in lattices. It has been selected as one of the standardizations in the NIST post-quantum cryptography project. Hardware-software co-design is a commonly adopted implementation strategy to address various implementation challenges, including limited resources, high performance, and flexibility requirements. In this study, we investigate using compact instruction set extensions (ISEs) for Dilithium, aiming to improve software efficiency with low hardware overheads. To begin with, we propose tightly coupled accelerators that are deeply integrated into the RISC-V processor. These accelerators target the most computationally demanding components in resource-constrained processors, such as polynomial generation, Number Theoretic Transform (NTT), and modular arithmetic. Next, we design a set of custom instructions that seamlessly integrate with the RISC-V base instruction formats, completing the accelerators in a compact manner. Subsequently, we implement our ISEs in a chip design for the Hummingbird E203 core and conduct performance benchmarks for Dilithium utilizing these ISEs. Additionally, we evaluate the resource consumption of the ISEs on FPGA and ASIC technologies. Compared to the reference software implementation on the RISC-V core, our co-design demonstrates a remarkable speedup factor ranging from 6.95 to 9.96. This significant improvement in performance is achieved by incorporating additional hardware resources, specifically, a 35% increase in LUTs, a 14% increase in FFs, 7 additional DSPs, and no additional RAM. Furthermore, compared to the state-of-the-art approach, our work achieves faster speed performance with a reduced circuit cost. Specifically, the usage of additional LUTs, FFs, and RAMs is reduced by 47.53%, 50.43%, and 100%, respectively. On ASIC technology, our approach demonstrates 12, 412 cell counts. Our co-design provides a better tradeoff implementation on speed performance and circuit overheads.

Pflow: An end-to-end heterogeneous acceleration framework for CNN inference on FPGAs

Field-Programmable Gate Arrays (FPGAs), renowned for their high performance per watt, are extensively utilized to accelerate Convolutional Neural Networks (CNNs) in edge computing environments, primarily employing dataflow-based and instruction set-based approaches. Compared to the instruction set-based approach that features fast and versatile circuit design, the dataflow-based approach can significantly enhance performance at the expense of design versatility. Nevertheless, edge computing environments require both high energy efficiency and adaptability to various scenarios. This paper proposes a novel end-to-end heterogeneous acceleration framework for CNN inference on FPGAs, named Pflow. The basic idea is to decouple network deployment and hardware details with a hardware–software co-design approach. First, a dataflow accelerator with an adaptive scheduling strategy is proposed. The adaptive scheduling strategy, along with a scalable design, maximizes hardware utilization in terms of computing resources and bandwidth. Secondly, we design a novel operator-perception method to automate the processes of network reconstruction and operator fusion. Thirdly, we integrate Pflow into the industrial-grade deep learning framework Paddle-Lite. We evaluate Pflow by implementing several networks on two representative FPGA platforms. Experimental results demonstrate that Pflow achieves energy efficiencies of 46.5 GOPS/W on Xilinx Zynq Ultrascale+ MPSoC 3EG and 59.4 GOPS/W on Virtex UltraScale+ XCVU13P. It also reaches a throughput of up to 255.7 GOPS on the former and 3.686 TOPS on the latter.

Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability.

While Siamese object tracking has witnessed significant advancements, its hard real-time behaviour on embedded devices remains inadequately addressed. In many application cases, an embedded implementation should not only have a minimal execution latency, but this latency should ideally also have zero variance, i.e., be predictable. This study aims to address this issue by meticulously analysing real-time predictability across different components of a deep-learning-based video object tracking system. Our detailed experiments not only indicate the superiority of Field-Programmable Gate Array (FPGA) implementations in terms of hard real-time behaviour but also unveil important time predictability bottlenecks. We introduce dedicated hardware accelerators for key processes, focusing on depth-wise cross-correlation and padding operations, utilizing high-level synthesis (HLS). Implemented on a KV260 board, our enhanced tracker exhibits not only a speed up, with a factor of 6.6, in mean execution time but also significant improvements in hard real-time predictability by yielding 11 times less latency variation as compared to our baseline. A subsequent analysis of power consumption reveals our approach's contribution to enhanced power efficiency. These advancements underscore the crucial role of hardware acceleration in realizing time-predictable object tracking on embedded systems, setting new standards for future hardware-software co-design endeavours in this domain.

Cross-layer transmission realized by light-emitting memristor for constructing ultra-deep neural network with transfer learning ability

Deep neural networks have revolutionized several domains, including autonomous driving, cancer detection, and drug design, and are the foundation for massive artificial intelligence models. However, hardware neural network reports still mainly focus on shallow networks (2 to 5 layers). Implementing deep neural networks in hardware is challenging due to the layer-by-layer structure, resulting in long training times, signal interference, and low accuracy due to gradient explosion/vanishing. Here, we utilize negative ultraviolet photoconductive light-emitting memristors with intrinsic parallelism and hardware-software co-design to achieve electrical information’s optical cross-layer transmission. We propose a hybrid ultra-deep photoelectric neural network and an ultra-deep super-resolution reconstruction neural network using light-emitting memristors and cross-layer block, expanding the networks to 54 and 135 layers, respectively. Further, two networks enable transfer learning, approaching or surpassing software-designed networks in multi-dataset recognition and high-resolution restoration tasks. These proposed strategies show great potential for high-precision multifunctional hardware neural networks and edge artificial intelligence.

Neuro-Symbolic Computing: Advancements and Challenges in Hardware–Software Co-Design

Neuro-Symbolic Computing: Advancements and Challenges in Hardware–Software Co-Design