Hardware Accelerators Research Articles

PEak: A Single Source of Truth for Hardware Design and Verification

Domain-specific languages for hardware can significantly enhance designer productivity, but sometimes at the cost of ease of verification. On the other hand, ISA specification languages are too static to be used during early stage design space exploration. We present PEak, an open-source hardware design and specification language, which aims to improve both design productivity and verification capability. PEak does this by providing a single source of truth for functional models, formal specifications, and RTL. PEak has been used in several academic projects, and PEak-generated RTL has been included in three fabricated hardware accelerators. In these projects, the formal capabilities of PEak were crucial for enabling both novel design space exploration techniques and automated compiler synthesis.

A survey of graph convolutional networks (GCNs) in FPGA-based accelerators

AbstractThis survey overviews recent Graph Convolutional Networks (GCN) advancements, highlighting their growing significance across various tasks and applications. It underscores the need for efficient hardware architectures to support the widespread adoption and development of GCNs, particularly focusing on platforms like FPGAs known for their performance and energy efficiency. This survey also outlines the challenges in deploying GCNs on hardware accelerators and discusses recent efforts to enhance efficiency. It encompasses a detailed review of the mathematical background of GCNs behind inference and training, a comprehensive review of recent works and architectures, and a discussion on performance considerations and future directions.

Optimizing embedded AI systems for autonomous driving: Challenges and solutions using bayesian networks

Abstract. Embedded Artificial Intelligence (AI) systems are important components of autonomous vehicles. However, incorporating AI into autonomous vehicles is technically complex, due to the constraints of computation, real-time processing of data, uncertainty handling, and hardware limitations. Bayesian Networks (BNs) are a promising method that allows probabilistic modelling in adaptive learning and environment perception. Here we report on an overview of the application of BNs on autonomous driving, with an emphasis on how BNs can be optimized for embedded system resource constraints, including both computational and energy. Various optimization techniques are discussed, such as model pruning, approximation, acceleration using hardware accelerators, such as Field-Programmable Gate Arrays (FPGA) and Application Specific Integrated Circuit (ASICs), and advanced cooling and power management to ensure AI reliability under high computational load. By reviewing these approaches, we aim to contribute to the development of more robust and green AI systems for autonomous driving.

Artificial intelligence in the Internet of Things: Integrating and optimizing AI algorithms for real-time data processing and decision-making

Abstract. The introduction of Artificial Intelligence (AI) will provide great opportunities and pose significant challenges in Internet of Things (IoT) systems. IoT devices can create huge amount of data in real-time. Meanwhile, it is crucial to handle immediate processing and decision-making in IoT applications. However, IoT devices are usually constrained by limited computation power, memory, and energy compared to traditional devices, making the deployment of efficient AI algorithms a challenging task. In this paper, we present different algorithmic strategies to overcome the above problems, including model compression, quantization, and hardware accelerators. On the other hand, we also focus on the role of decentralization and edge computing architectures to increase the scalability and performance of AI-IoT systems. Finally, this paper also reviews energy-efficient AI algorithms and latency reduction methods to make the decision process real-time for various IoT applications.

Benchmarking Big Data Systems: Performance and Decision-Making Implications in Emerging Technologies

Systems for graph processing are a key enabler for insights from large-scale graphs that are critical to many new advanced technologies such as Artificial Intelligence, Internet of Things, and blockchain. In this study, we benchmark another two widely utilized graph processing systems, Apache Spark GraphX and Apache Fink, concerning the key performance criterion by means of response time, scalability, and computational complexity. We demonstrate our results which show the capability of each system for real-world graph applications, and hence, providing a quantitative understanding to select the system for our purpose. GraphX’s strength was in processing batch in-memory workloads typical of blockchain and machine learning model optimization, while Flink excelled in processing stream data, which is timely and important to the IoT world. These performance characteristics emphasize how the capabilities of graph processing systems can match the requirements for the performance of different emerging technology applications. Our findings ultimately inform practitioners about system efficiencies and limitations, but also the recent advances in hardware accelerators and algorithmic improvements aimed at shaping the new graph processing frontier in diverse technology domains.

Closing the gap between SGP4 and high-precision propagation via differentiable programming

The simplified general perturbations 4 (SGP4) orbital propagation model is one of the most widely used methods for rapidly and reliably predicting the positions and velocities of objects orbiting Earth. Over time, SGP models have undergone refinement to enhance their efficiency and accuracy. Nevertheless, they still do not match the precision offered by high-precision numerical propagators, which can predict the positions and velocities of space objects in low-Earth orbit with significantly smaller errors.In this study, we introduce a novel differentiable version of SGP4, named ∂SGP4. By porting the source code of SGP4 into a differentiable program based on PyTorch, we unlock a whole new class of techniques enabled by differentiable orbit propagation, including spacecraft orbit determination, state conversion, covariance similarity transformation, state transition matrix computation, and covariance propagation. Besides differentiability, our ∂SGP4 supports parallel propagation of a batch of two-line elements (TLEs) in a single execution and it can harness modern hardware accelerators like GPUs or XLA devices (e.g. TPUs) thanks to running on the PyTorch backend.Furthermore, the design of ∂SGP4 makes it possible to use it as a differentiable component in larger machine learning (ML) pipelines, where the propagator can be an element of a larger neural network that is trained or fine-tuned with data. Consequently, we propose a novel orbital propagation paradigm, ML-∂SGP4. In this paradigm, the orbital propagator is enhanced with neural networks attached to its input and output. Through gradient-based optimization, the parameters of this combined model can be iteratively refined to achieve precision surpassing that of SGP4. Fundamentally, the neural networks function as identity operators when the propagator adheres to its default behavior as defined by SGP4. However, owing to the differentiability ingrained within ∂SGP4, the model can be fine-tuned with ephemeris data to learn corrections to both inputs and outputs of SGP4. This augmentation enhances precision while maintaining the same computational speed of ∂SGP4 at inference time. This paradigm empowers satellite operators and researchers, equipping them with the ability to train the model using their specific ephemeris or high-precision numerical propagation data.

A Simulation and Experimental Platform for Communication Courses Based on Digital Signal Processor

In this paper, a simulation and experimental platform is designed and implemented to overcome the shortcomings of the existing experimental platform for communication related courses. Firstly, the overall scheme of the simulation and experimental platform are introduced, including the specific hardware composition and supporting software. Secondly, the implement details are described in the following order: system composition, peripherals, hardware accelerator, and interface to the host computer. The main system is based on an independently developed digital signal processor, which may relieve the bottleneck problem of imported chips. Using FPGA to implement peripherals enhances the flexibility of the entire system and provides the ability for further development. The feasibility of reducing the cost is then discussed. Thirdly, the design and implementation of the simulator, compiler and assembler, debugger and other tools are introduced respectively. The simulator can operate under normal mode and debugging mode. The former can be used for ordinary experiments for teaching purpose, while the latter is for further development of graduated students. Finally, the prepared experimental projects are shown, including 6 basic experiments and 6 communication algorithm experiments. With the help of these experiments, students can better understand the principles and implementation of corresponding communication algorithms, and experience how different parameters may impact on the communication results, by modifying the configuration of the experimental system. The demonstrations of prototype prove that the simulation and experimental platform can meet the needs of communication related courses. The future work includes developing more experimental projects, raising integration to reduce the costs, and improving the entire system based on student feedback.

Towards high-performance deep learning architecture and hardware accelerator design for robust analysis in diffuse correlation spectroscopy

This study proposes a compact deep learning (DL) architecture and a highly parallelized computing hardware platform to reconstruct the blood flow index (BFi) in diffuse correlation spectroscopy (DCS). We leveraged a rigorous analytical model to generate autocorrelation functions (ACFs) to train the DL network. We assessed the accuracy of the proposed DL using simulated and milk phantom data. Compared to convolutional neural networks (CNN), our lightweight DL architecture achieves 66.7% and 18.5% improvement in MSE for BFi and the coherence factor β, using synthetic data evaluation. The accuracy of rBFi over different algorithms was also investigated. We further simplified the DL computing primitives using subtraction for feature extraction, considering further hardware implementation. We extensively explored computing parallelism and fixed-point quantization within the DL architecture. With the DL model's compact size, we employed unrolling and pipelining optimizations for computation-intensive for-loops in the DL model while storing all learned parameters in on-chip BRAMs. We also achieved pixel-wise parallelism, enabling simultaneous, real-time processing of 10 and 15 autocorrelation functions on Zynq-7000 and Zynq-UltraScale+ field programmable gate array (FPGA), respectively. Unlike existing FPGA accelerators that produce BFi and the β from autocorrelation functions on standalone hardware, our approach is an encapsulated, end-to-end on-chip conversion process from intensity photon data to the temporal intensity ACF and subsequently reconstructing BFi and β. This hardware platform achieves an on-chip solution to replace post-processing and miniaturize modern DCS systems that use single-photon cameras. We also comprehensively compared the computational efficiency of our FPGA accelerator to CPU and GPU solutions.

Comprehensive genome analysis and variant detection at scale using DRAGEN.

Research and medical genomics require comprehensive, scalable methods for the discovery of novel disease targets, evolutionary drivers and genetic markers with clinical significance. This necessitates a framework to identify all types of variants independent of their size or location. Here we present DRAGEN, which uses multigenome mapping with pangenome references, hardware acceleration and machine learning-based variant detection to provide insights into individual genomes, with ~30 min of computation time from raw reads to variant detection. DRAGEN outperforms current state-of-the-art methods in speed and accuracy across all variant types (single-nucleotide variations, insertions or deletions, short tandem repeats, structural variations and copy number variations) and incorporates specialized methods for analysis of medically relevant genes. We demonstrate the performance of DRAGEN across 3,202 whole-genome sequencing datasets by generating fully genotyped multisample variant call format files and demonstrate its scalability, accuracy and innovation to further advance the integration of comprehensive genomics. Overall, DRAGEN marks a major milestone in sequencing data analysis and will provide insights across various diseases, including Mendelian and rare diseases, with a highly comprehensive and scalable platform.

FedNIC: enhancing privacy-preserving federated learning via homomorphic encryption offload on SmartNIC

Federated learning (FL) has emerged as a promising paradigm for secure distributed machine learning model training across multiple clients or devices, enabling model training without having to share data across the clients. However, recent studies revealed that FL could be vulnerable to data leakage and reconstruction attacks even if the data itself are never shared with another client. Thus, to resolve such vulnerability and improve the privacy of all clients, a class of techniques, called privacy-preserving FL, incorporates encryption techniques, such as homomorphic encryption (HE), to encrypt and fully protect model information from being exposed to other parties. A downside to this approach is that encryption schemes like HE are very compute-intensive, often causing inefficient and excessive use of client CPU resources that can be used for other uses. To alleviate this issue, this study introduces a novel approach by leveraging smart network interface cards (SmartNICs) to offload compute-intensive HE operations of privacy-preserving FL. By employing SmartNICs as hardware accelerators, we enable efficient computation of HE while saving CPU cycles and other server resources for more critical tasks. In addition, by offloading encryption from the host to another device, the details of encryption remain secure even if the host is compromised, ultimately improving the security of the entire FL system. Given such benefits, this paper presents an FL system named FedNIC that implements the above approach, with an in-depth description of the architecture, implementation, and performance evaluations. Our experimental results demonstrate a more secure FL system with no loss in model accuracy and up to 25% in reduced host CPU cycle, but with a roughly 46% increase in total training time, showing the feasibility and tradeoffs of utilizing SmartNICs as an encryption offload device in federated learning scenarios. Finally, we illustrate promising future study and potential optimizations for a more secure and privacy-preserving federated learning system.

Efficient hardware accelerators for k-nearest neighbors classification using most significant digit first arithmetic

Efficient hardware accelerators for k-nearest neighbors classification using most significant digit first arithmetic

LDF-BNN: A Real-Time and High-Accuracy Binary Neural Network Accelerator Based on the Improved BNext.

Significant progress has been made in industrial defect detection due to the powerful feature extraction capabilities of deep neural networks (DNNs). However, the high computational cost and memory requirement of DNNs pose a great challenge to the deployment of industrial edge-side devices. Although traditional binary neural networks (BNNs) have the advantages of small storage space requirements, high parallel computing capability, and low power consumption, the problem of significant accuracy degradation cannot be ignored. To tackle these challenges, this paper constructs a BNN with layered data fusion mechanism (LDF-BNN) based on BNext. By introducing the above mechanism, it strives to minimize the bandwidth pressure while reducing the loss of accuracy. Furthermore, we have designed an efficient hardware accelerator architecture based on this mechanism, enhancing the performance of high-accuracy BNN models with complex network structures. Additionally, the introduction of multi-storage parallelism alleviates the limitations imposed by the internal transfer rate, thus improving the overall computational efficiency. The experimental results show that our proposed LDF-BNN outperforms other methods in the comprehensive comparison, achieving a high accuracy of 72.23%, an image processing rate of 72.6 frames per second (FPS), and 1826 giga operations per second (GOPs) on the ImageNet dataset. Meanwhile, LDF-BNN can also be well applied to defect detection dataset Mixed WM-38, achieving a high accuracy of 98.70%.

Hardware-accelerated neural network model for early prediction of sudden cardiac arrest based on heart rate variability metrics

AbstractSudden Cardiac Arrest (SCA) constitutes a dire medical condition, marked by the abrupt cessation of effective blood circulation due to the heart's failure to contract properly. This leads to acute circulatory collapse, often culminating in loss of consciousness within an hour and potentially resulting in fatality within minutes if left unattended. Heart rate variability (HRV) serves as a critical biometric, derived from electrocardiogram (ECG) signals with ventricular depolarization waves to further calculate the R-R Intervals (RRIs). These intervals provide the basis for extracting various characteristics of cardiac rhythm, encompassing time-domain, frequency-domain, and nonlinear features. This study presents a neural network-based classification algorithm that leverages HRV metrics to categorize patients into SCA and Normal Sinus Rhythm (NSR) cohorts. The proposed neural network (NN) model showcased impressive results, achieving an accuracy of 96.23%, a sensitivity of 94.35%, and a specificity of 98.12% in detecting SCA, as evaluated through leave-one-subject-out analysis. In order to harness the benefits of hardware acceleration, the algorithm is implemented on a Field-Programmable Gate Array (FPGA). Its computational efficiency is subsequently benchmarked against traditional software-based methodologies. The hardware-level implementation is made possible in Verilog hardware description language (HDL) and was verified successfully with expected performance by register-transfer level (RTL) simulation via Vivado 2020.2.

Enhancing 2-D Electrical Impedance Tomography Throughput With a Combined FPGA and Edge GPU-Based Hardware Accelerator

Enhancing 2-D Electrical Impedance Tomography Throughput With a Combined FPGA and Edge GPU-Based Hardware Accelerator

Efficient memristor accelerator for transformer self-attention functionality

The adoption of transformer networks has experienced a notable surge in various AI applications. However, the increased computational complexity, stemming primarily from the self-attention mechanism, parallels the manner in which convolution operations constrain the capabilities and speed of convolutional neural networks (CNNs). The self-attention algorithm, specifically the matrix-matrix multiplication (MatMul) operations, demands a substantial amount of memory and computational complexity, thereby restricting the overall performance of the transformer. This paper introduces an efficient hardware accelerator for the transformer network, leveraging memristor-based in-memory computing. The design targets the memory bottleneck associated with MatMul operations in the self-attention process, utilizing approximate analog computation and the highly parallel computations facilitated by the memristor crossbar architecture. Remarkably, this approach resulted in a reduction of approximately 10 times in the number of multiply-accumulate (MAC) operations in transformer networks, while maintaining 95.47% accuracy for the MNIST dataset, as validated by a comprehensive circuit simulator employing NeuroSim 3.0. Simulation outcomes indicate an area utilization of 6895.7 μm2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu m^2$$\\end{document}, a latency of 15.52 seconds, an energy consumption of 3 mJ, and a leakage power of 59.55 μW\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu W$$\\end{document}. The methodology outlined in this paper represents a substantial stride towards a hardware-friendly transformer architecture for edge devices, poised to achieve real-time performance.

Brief Study on Convolutional Neural Networks

Convolutional Neural Networks (CNNs) have revolutionized the field of computer vision and pattern recognition, offering state-of-the-art performance in tasks such as image classification, object detection, and segmentation. This study explores the architecture, training strategies, and applications of CNNs, focusing on their ability to automatically learn spatial hierarchies of features through layers of convolutional filters. The research evaluates various CNN architectures, including Lent, Alex Net, and ResNet, highlighting their improvements in accuracy and efficiency. Additionally, the study investigates advanced techniques such as data augmentation, transfer learning, and regularization methods (e. g, dropout and batch normalization), which enhance the model's generalization capabilities. Through empirical experiments, we demonstrate the effectiveness of CNNs in real-world scenarios, including medical image analysis, autonomous driving, and facial recognition. The study concludes with a discussion of the future trends of CNNs, particularly in the context of deep learning optimization, hardware acceleration, and integration with emerging technologies like edge computing and quantum machine learning.

A Survey on Architectures, Hardware Acceleration and Challenges for In-Network Computing

A Survey on Architectures, Hardware Acceleration and Challenges for In-Network Computing

Hardware Acceleration for High-Volume Operations of CRYSTALS-Kyber and CRYSTALS-Dilithium

Many high-demand digital services need to perform several cryptographic operations, such as key exchange or security credentialing, in a concise amount of time. In turn, the security of some of these cryptographic schemes is threatened by advances in quantum computing, as quantum computer could break their security in the near future. Post-quantum cryptography (PQC) is an emerging field that studies cryptographic algorithms that resist such attacks. The National Institute of Standards and Technology (NIST) has selected the CRYSTALS-Kyber Key Encapsulation Mechanism and the CRYSTALS-Dilithium Digital Signature algorithm as primary PQC standards. In this article, we present field-programmable gate array (FPGA)-based hardware accelerators for high-volume operations of both schemes. We apply high-level synthesis (HLS) for hardware optimization, leveraging a batch processing approach to maximize the memory throughput and applying custom HLS logic to specific algorithmic components. Using reconfigurable FPGAs, we show that our hardware accelerators achieve speedups between 3 \(\times\) and 9 \(\times\) over software baseline implementations, even over ones leveraging CPU vector architectures. Furthermore, the methods used in this study can also be extended to the new CRYSTALS-based NIST FIPS drafts, ML-KEM and ML-DSA, with similar acceleration results.

Probabilistic computing enabled by continuous random numbers sampled from in-plane magnetized stochastic magnetic tunnel junctions

Hardware acceleration of probabilistic computing has recently attracted significant attention in the slowing down of Moore's law. A randomly fluctuating bit called as p-bit constitutes a fundamental building block for this type of physics-inspired computing scheme, which can be efficiently built out of emerging devices. Here, we report a probabilistic computing set-up, where random numbers are sampled from stochastic magnetic tunnel junctions with in-plane magnetic anisotropy. Although the sampled data have largely bipolar-like probability distributions compared to the ideally uniform ones, the results show a reasonable performance in a standard simulated annealing process on Boolean satisfiability problems up to 100 variables. The systematic simulations suggest the importance of probability distribution where some additional intermediate states help to increase the performance.

Hardware acceleration of Tiny YOLO deep neural networks for sign language recognition: A comprehensive performance analysis

In this paper, we benchmark two automation frameworks, Vitis AI and FINN, for sign language recognition on a Field Programmable Gate Array (FPGA). We conducted an in-depth exploration of both frameworks using Tiny YOLOv2 networks by varying design parameters such as precision, parallelism ratio, etc. Further, a fair baseline comparison is made based on accuracy, speed, and hardware resources. Experimental findings demonstrate that the Vitis AI outperforms the FINN framework and traditional GPU and CPU platforms by achieving significant improvements of 1.08x, 1.7x, and 2.9x in terms of latency. Leveraging Vitis AI, our system achieved a detection speed of 32.7 frames per second (FPS) on the Kria KV260 FPGA with a power consumption rate of 5.6 W and an impressive mean Average Precision (mAP) score of 61.2% on the Hindi Indian Sign Language (ISL) dataset.