Efficient Hardware Accelerator Research Articles

Efficient hardware accelerators for k-nearest neighbors classification using most significant digit first arithmetic

Efficient hardware accelerators for k-nearest neighbors classification using most significant digit first arithmetic

LDF-BNN: A Real-Time and High-Accuracy Binary Neural Network Accelerator Based on the Improved BNext.

Significant progress has been made in industrial defect detection due to the powerful feature extraction capabilities of deep neural networks (DNNs). However, the high computational cost and memory requirement of DNNs pose a great challenge to the deployment of industrial edge-side devices. Although traditional binary neural networks (BNNs) have the advantages of small storage space requirements, high parallel computing capability, and low power consumption, the problem of significant accuracy degradation cannot be ignored. To tackle these challenges, this paper constructs a BNN with layered data fusion mechanism (LDF-BNN) based on BNext. By introducing the above mechanism, it strives to minimize the bandwidth pressure while reducing the loss of accuracy. Furthermore, we have designed an efficient hardware accelerator architecture based on this mechanism, enhancing the performance of high-accuracy BNN models with complex network structures. Additionally, the introduction of multi-storage parallelism alleviates the limitations imposed by the internal transfer rate, thus improving the overall computational efficiency. The experimental results show that our proposed LDF-BNN outperforms other methods in the comprehensive comparison, achieving a high accuracy of 72.23%, an image processing rate of 72.6 frames per second (FPS), and 1826 giga operations per second (GOPs) on the ImageNet dataset. Meanwhile, LDF-BNN can also be well applied to defect detection dataset Mixed WM-38, achieving a high accuracy of 98.70%.

Efficient memristor accelerator for transformer self-attention functionality

The adoption of transformer networks has experienced a notable surge in various AI applications. However, the increased computational complexity, stemming primarily from the self-attention mechanism, parallels the manner in which convolution operations constrain the capabilities and speed of convolutional neural networks (CNNs). The self-attention algorithm, specifically the matrix-matrix multiplication (MatMul) operations, demands a substantial amount of memory and computational complexity, thereby restricting the overall performance of the transformer. This paper introduces an efficient hardware accelerator for the transformer network, leveraging memristor-based in-memory computing. The design targets the memory bottleneck associated with MatMul operations in the self-attention process, utilizing approximate analog computation and the highly parallel computations facilitated by the memristor crossbar architecture. Remarkably, this approach resulted in a reduction of approximately 10 times in the number of multiply-accumulate (MAC) operations in transformer networks, while maintaining 95.47% accuracy for the MNIST dataset, as validated by a comprehensive circuit simulator employing NeuroSim 3.0. Simulation outcomes indicate an area utilization of 6895.7 μm2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu m^2$$\\end{document}, a latency of 15.52 seconds, an energy consumption of 3 mJ, and a leakage power of 59.55 μW\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu W$$\\end{document}. The methodology outlined in this paper represents a substantial stride towards a hardware-friendly transformer architecture for edge devices, poised to achieve real-time performance.

Exploiting beam search confidence for energy-efficient speech recognition

With mobile and embedded devices getting more integrated in our daily lives, the focus is increasingly shifting toward human-friendly interfaces, making automatic speech recognition (ASR) a central player as the ideal means of interaction with machines. ASR is essential for many cognitive computing applications, such as speech-based assistants, dictation systems and real-time language translation. Consequently, interest in speech technology has grown in the last few years, with more systems being proposed and higher accuracy levels being achieved, even surpassing human accuracy. However, highly accurate ASR systems are computationally expensive, requiring on the order of billions of arithmetic operations to decode each second of audio, which conflicts with a growing interest in deploying ASR on edge devices. On these devices, efficient hardware acceleration is key for achieving acceptable performance. In this paper, we propose a technique to improve the energy efficiency and performance of ASR systems, focusing on low-power hardware for edge devices. We focus on optimizing the DNN-based acoustic model evaluation, as we have observed it to be the main bottleneck in popular ASR systems, by leveraging run-time information from the beam search. By doing so, we reduce energy and execution time of the acoustic model evaluation by 25.6 and 25.9 %, respectively, with negligible accuracy loss.

Revolutionising Ai Deployment: Survey On Hardware Acceleration for Machine Learning Optimisation and Resilience

This compilation of research studies holds the utmost significance in hardware acceleration for machine learning. In our current era, characterised by the exponential growth of artificial intelligence (AI) applications, these studies tackle crucial challenges in optimising neural network accelerators' performance, energy efficiency, and resilience. The importance lies in their potential to revolutionise AI implementation across various domains. Efficient hardware accelerators are a cornerstone in unlocking the full potential of AI, enabling breakthroughs in deep learning, high-speed train fault detection and isolation, and numerous other applications. By improving memory management, facts placement, bus scheduling, and fault tolerance, that research paves the way for AI structures which are both powerful and sustainable, making AI accessible and impactful in a wide variety of fields. This research is important for fostering the growth and adoption of AI, ultimately remodelling how we interact with technology and facts in our daily lives.

Design and Hardware Implementation of CNN-GCN Model for Skeleton-Based Human Action Recognition

Deep learning algorithms in general have shown outstanding performances on tasks involving Human Action Recognition and spatio-temporal modeling. In our ever-more interconnected world where mobile and edge devices take center stage, there is a growing need for algorithms to operate directly on embedded platforms. Field-Programmable Gate Arrays (FPGA), owing to their reprogrammable nature and low-power attributes, stand out as excellent choices for these edge computing applications. In this work, our aims are threefold. Firstly, we aim to design and develop a novel custom model that combines Convolutional Neural Networks (CNNs) and Graph Convolutional Networks (GCNs) to effectively capture spatial and temporal features from skeleton data to achieve Human Action Recognition. Secondly, we endeavor to implement this custom model onto FPGA hardware using Vitis-AI, thus exploring the potential for efficient hardware acceleration of deep learning models. Lastly, we seek to evaluate the real-time performance of the FPGA-accelerated model in comparison to CPU and GPU implementations, assessing its suitability for deployment in real-world applications requiring low-latency inference. Experimental results based on the NTU dataset demonstrate the efficiency and accuracy of our custom model compared to traditional CPU or GPU implementations.

An efficient hardware accelerator for NTT-based polynomial multiplication using FPGA

An efficient hardware accelerator for NTT-based polynomial multiplication using FPGA

Leveraging Seed Generation for Efficient Hardware Acceleration of Lossless Compression of Remotely Sensed Hyperspectral Images

In the field of satellite imaging, effectively managing the enormous volumes of data from remotely sensed hyperspectral images presents significant challenges due to the limited bandwidth and power available in spaceborne systems. In this paper, we describe the hardware acceleration of a highly efficient lossless compression algorithm, specifically designed for real-time hyperspectral image processing on FPGA platforms. The algorithm utilizes an innovative seed generation method for square root calculations to significantly boost data throughput and reduce energy consumption, both of which represent key factors in satellite operations. When implemented on the Cyclone V FPGA, our method achieves a notable operational throughput of 1598.67 Mega Samples per second (MSps) and maintains a power requirement of under 1 Watt, leading to an efficiency rate of 1829.1 MSps/Watt. A comparative analysis with existing and related state-of-the-art implementations confirms that our system surpasses conventional performance standards, thus facilitating the efficient processing of large-scale hyperspectral datasets, especially in environments where throughput and low energy consumption are prioritized.

Review of Advanced Methods in Hardware Acceleration for Deep Neural Networks

Abstract: Convolutional neural networks have become very efficient in performing tasks like Object Detection providing human like accuracy. However, their practical implementation needs significant hardware resources and memory bandwidth. In recent past a lot of research is being carried out for achieving higher efficiency in implementing such neural networks in hardware. We talk about FPGAs for hardware implementation due to their flexibility for customisation for such neural network architectures. In this paper we will discuss the metrics for efficient hardware accelerator and general methods available for achieving an efficient design. Further, we will discuss the actual methods used by recent research for implementation of deep neural networks particularly for object detection related applications. These methods range from actual ASIC design like TPUs [1] for on chip acceleration, state of the art open source designs like Gemini to methods like hardware reuse, re-configurable nodes and approximation in computations as a trade-off between speed and accuracy. This paper will be a valuable summary for the researchers starting in the field of hardware accelerators design for neural networks

Reconfigurable stochastic neurons based on strain engineered low barrier nanomagnets

Stochastic neurons are efficient hardware accelerators for solving a large variety of combinatorial optimization problems. ‘Binary’ stochastic neurons (BSN) are those whose states fluctuate randomly between two levels +1 and −1, with the probability of being in either level determined by an external bias. ‘Analog’ stochastic neurons (ASNs), in contrast, can assume any state between the two levels randomly (hence ‘analog’) and can perform analog signal processing. They may be leveraged for such tasks as temporal sequence learning, processing and prediction. Both BSNs and ASNs can be used to build efficient and scalable neural networks. Both can be implemented with low (potential energy) barrier nanomagnets (LBMs) whose random magnetization orientations encode the binary or analog state variables. The difference between them is that the potential energy barrier in a BSN LBM, albeit low, is much higher than that in an ASN LBM. As a result, a BSN LBM has a clear double well potential profile, which makes its magnetization orientation assume one of two orientations at any time, resulting in the binary behavior. ASN nanomagnets, on the other hand, hardly have any energy barrier at all and hence lack the double well feature. That makes their magnetizations fluctuate in an analog fashion. Hence, one can reconfigure an ASN to a BSN, and vice-versa, by simply raising and lowering the energy barrier. If the LBM is magnetostrictive, then this can be done with local (electrically generated) strain. Such a reconfiguration capability heralds a powerful field programmable architecture for a p-computer whereby hardware for very different functionalities such as combinatorial optimization and temporal sequence learning can be integrated in the same substrate in the same processing run. This is somewhat reminiscent of heterogeneous integration, except this is integration of functionalities or computational fabrics rather than components. The energy cost of reconfiguration is miniscule. There are also other applications of strain mediated barrier control that do not involve reconfiguring a BSN to an ASN or vice versa, e.g. adaptive annealing in energy minimization computing (Boltzmann or Ising machines), emulating memory hierarchy in a dynamically reconfigurable fashion, and control over belief uncertainty in analog stochastic neurons. Here, we present a study of strain engineered barrier control in unconventional computing.

A Progressive Subnetwork Searching Framework for Dynamic Inference.

Deep neural network (DNN) model compression is a popular and important optimization method for efficient and fast hardware acceleration. However, the compressed model is usually fixed, without the capability to tune the computing complexity (i.e., latency in hardware) on-the-fly, depending on dynamic latency requirements, workloads, and computing hardware resource allocation. To address this challenge, dynamic DNN with run-time adaption of computing structures has been constructed through training with a cross-entropy objective function consisting of multiple subnets sampled from the supernet. Our investigations in this work show that the performance of dynamic inference highly relies on the quality of subnet sampling. To construct a dynamic DNN with multiple high-quality subnets, we propose a progressive subnetwork searching framework, which is embedded with several proposed new techniques, including trainable noise ranking, channel-group sampling, selective fine-tuning, and subnet filtering. Our proposed framework empowers the target dynamic DNN with higher accuracy for all the subnets compared with prior works on both the Canadian Institute for Advanced Research dataset with 10 classes (CIFAR-10) and ImageNet datasets. Specifically, compared with United States-Neural Network (US-NN), our method achieves 0.9% average accuracy gain for Alexnet, 2.5% for ResNet18, 1.1% for Visual Geometry Group (VGG)11, and 0.58% for MobileNetv1, on the ImageNet dataset, respectively. Moreover, to demonstrate run-time tuning of computing latency of dynamic DNN in real computing system, we have deployed our constructed dynamic networks into Nvidia Titan graphics processing unit (GPU) and Intel Xeon central processing unit (CPU), showing great improvement over prior works. The code is available at https://github.com/ASU-ESIC-FAN-Lab/Dynamic-inference.

Hardware acceleration of DNA pattern matching using analog resistive CAMs

DNA pattern matching is essential for many widely used bioinformatics applications. Disease diagnosis is one of these applications since analyzing changes in DNA sequences can increase our understanding of possible genetic diseases. The remarkable growth in the size of DNA datasets has resulted in challenges in discovering DNA patterns efficiently in terms of run time and power consumption. In this paper, we propose an efficient pipelined hardware accelerator that determines the chance of the occurrence of repeat-expansion diseases using DNA pattern matching. The proposed design parallelizes the DNA pattern matching task using associative memory realized with analog content-addressable memory and implements an algorithm that returns the maximum number of consecutive occurrences of a specific pattern within a DNA sequence. We fully implement all the required hardware circuits with PTM 45-nm technology, and we evaluate the proposed architecture on a practical human DNA dataset. The results show that our design is energy-efficient and accelerates the DNA pattern matching task by more than 100× compared to the approaches described in the literature.

Efficient number theoretic transform accelerator for CRYSTALS-Kyber

<p>The national institute of standards and technology (NIST) has presented its draft of the module-lattice-based key-encapsulation mechanism standard (MLBKEMS), choosing cryptographic suite for algebraic lattices (CRYSTALS)- Kyber as the base encryption. Existing hardware implementations of modern cryptography will need to process the new standard efficiently. The primary process in CRYSTALS-Kyber key-encapsulation mechanism (KEM) is the number theoretic transform (NTT), which requires heavy computing power. This paper contributes an efficient hardware accelerator for NTT and inverse NTT (INTT) by CRYSTAL-Kyber parameters. The proposed design utilizes the K-RED algorithm for reducing polynomial multiplication. It also incorporates the BrentKung method for efficient modular addition and subtraction operation with an address generator to control the sequences of computation. On the Xilinx Artix 7 field programmable gate array (FPGA), our design achieves 262 MHz clock speed, utilizing only 1405 LUTs.</p>

Reinforcement Learning-Driven Bit-Width Optimization for the High-Level Synthesis of Transformer Designs on Field-Programmable Gate Arrays

With the rapid development of deep-learning models, especially the widespread adoption of transformer architectures, the demand for efficient hardware accelerators with field-programmable gate arrays (FPGAs) has increased owing to their flexibility and performance advantages. Although high-level synthesis can shorten the hardware design cycle, determining the optimal bit-width for various transformer designs remains challenging. Therefore, this paper proposes a novel technique based on a predesigned transformer hardware architecture tailored for various types of FPGAs. The proposed method leverages a reinforcement learning-driven mechanism to automatically adapt and optimize bit-width settings based on user-provided transformer variants during inference on an FPGA, significantly alleviating the challenges related to bit-width optimization. The effect of bit-width settings on resource utilization and performance across different FPGA types was analyzed. The efficacy of the proposed method was demonstrated by optimizing the bit-width settings for users’ transformer-based model inferences on an FPGA. The use of the predesigned hardware architecture significantly enhanced the performance. Overall, the proposed method enables effective and optimized implementations of user-provided transformer-based models on an FPGA, paving the way for edge FPGA-based deep-learning accelerators while reducing the time and effort typically required in fine-tuning bit-width settings.

A Compact and Efficient Hardware Accelerator for RNS-CKKS En/Decoding and En/Decryption

A Compact and Efficient Hardware Accelerator for RNS-CKKS En/Decoding and En/Decryption

An efficient hardware accelerator for monotonic graph algorithms on dynamic directed graphs

随着现实世界中动态图计算需求的快速增长, 现有的研究工作已经提出了多种方法来有效支持单调图算法在动态有向图中的处理. 然而, 由于动态有向图的图结构频繁发生变化，其相邻图顶点之间的状态更新存在复杂的依赖关系, 这使得现有的软硬件方法在处理单调图算法时依然面临着数据访问成本高和收敛速度慢的问题. 为此, 本文提出了一种面向动态有向图的单调图算法加速器DSGraph, 它能够充分利用图顶点之间的依赖关系来加快单调图算法在动态有向图处理中的收敛速度, 并有效降低数据访问成本. 具体来说, DSGraph 通过实时提取动态有向图中图顶点的局部拓扑依赖顺序来执行异步迭代处理, 从而显著减少冗余的图顶点状态更新. 同时, DSGraph 设计了一种异步迭代流水线架构, 其按照依赖顺序对图顶点状态进行异步迭代处理, 从而加速图顶点状态传播速度并减少数据访问开销. 最后, DSGraph 提出了一种无阻塞数据同步机制, 通过并行执行本地图顶点的状态更新和外部图顶点的数据同步来减少系统同步开销. 实验显示, 与目前最先进的面向单调图算法的动态图处理系统KickStarter 相比, DSGraph 将动态有向图处理速度平均提升了11.2 倍.

Convolutional Neural Networks using FPGA-based Pipelining

In order to speed up convolutional neural networks (CNNs), this study gives a complete overview of the use of FPGA-based pipelining for hardware acceleration of CNNs. These days, most people use convolutional neural networks (CNNs) to perform computer vision tasks like picture categorization and object recognition. The processing and memory demands of CNNs, however, can be excessive, especially for real-time applications. In order to speed up CNNs, FPGA-based pipelining has emerged as a viable option thanks to its parallel processing capabilities and low power consumption. The examination describes the fundamentals of FPGA-based pipelining and the basic structure of convolutional neural networks (CNNs). The current best practises for developing pipelined accelerators for CNNs on FPGAs are then reviewed, covering topics like partitioning and pipelining. Area and power limits, memory needs, and latency considerations are only some of the difficulties and trade-offs discussed in the article. In addition, the survey evaluates and contrasts the various pipelined FPGA accelerators for CNNs in terms of performance, energy consumption, and resource utilisation. Future directions and potential research areas are also discussed in the paper, such as the use of approximate computing techniques, the integration of reconfigurable architectures with emerging memory technologies, and the exploration of hybrid architectures that combine FPGAs and other hardware accelerators. This survey was created to aid researchers and practitioners in developing efficient and effective hardware accelerators for neural networks by providing a thorough overview of current trends and issues in FPGA-based pipelining for CNNs.

A Unified Point Multiplication Architecture of Weierstrass, Edward and Huff Elliptic Curves on FPGA

This article presents an area-aware unified hardware accelerator of Weierstrass, Edward, and Huff curves over GF(2233) for the point multiplication step in elliptic curve cryptography (ECC). The target implementation platform is a field-programmable gate array (FPGA). In order to explore the design space between processing time and various protection levels, this work employs two different point multiplication algorithms. The first is the Montgomery point multiplication algorithm for the Weierstrass and Edward curves. The second is the Double and Add algorithm for the Binary Huff curve. The area complexity is reduced by efficiently replacing storage elements that result in a 1.93 times decrease in the size of the memory needed. An efficient Karatsuba modular multiplier hardware accelerator is implemented to compute polynomial multiplications. We utilized the square arithmetic unit after the Karatsuba multiplier to execute the quad-block variant of a modular inversion, which preserves lower hardware resources and also reduces clock cycles. Finally, to support three different curves, an efficient controller is implemented. Our unified architecture can operate at a maximum of 294 MHz and utilizes 7423 slices on Virtex-7 FPGA. It takes less computation time than most recent state-of-the-art implementations. Thus, combining different security curves (Weierstrass, Edward, and Huff) in a single design is practical for applications that demand different reliability/security levels.

Algorithm and hardware co-design co-optimization framework for LSTM accelerator using quantized fully decomposed tensor train

Nowadays, from companies to academics, researchers across the world are interested in developing Deep Neural Networks (DNNs) due to their incredible feats in various applications, such as image recognition, playing complex games, and large-scale information retrieval such as web search. However, when people enjoy the advantages of DNNs, the high computational and power demands on resource-constrained electronic devices of the DNN model have received more attention. Optimizing the DNN model, such as model compression, is crucial to ensure wide deployment of DNNs and promote DNNs to implement most resource-constrained scenarios. Among many techniques, tensor train (TT) decomposition is considered a very promising technology. Although our previous efforts achieve (1) expanding limits of the number of multiplications eliminating all redundant computations; and (2) decomposing into multistage processing to reduce memory traffic, the potential of this work is not fully explored.In this paper, we investigate and demystify the TT decomposition within a thoughtful hardware mind. This paper develops an efficient hardware optimization methodology within a novel hardware solution. Two key merits will be achieved in this project: (1) it enables a novel approach to apply TT-decomposition on the entire LSTM model; (2) a much more efficient quantization method has been proposed at the hardware optimization level; (3) an efficient hardware accelerator that unitizes the hardware and algorithm co-design method has been designated.Based on these novelties, the proposed work can achieve 1.69× power reduction and 2.28× power efficiency (GOPS/W) in different workloads. In addition, compared to the state-of-the-art C-LSTM, it achieves 2.09× higher throughput, 3.67% accuracy increase, 2.45× power efficiency, and a 1.18× power reduction. The results show that our proposed accelerator exhibits significant advantages over state-of-the-art solutions.

Large Field-Size Throughput/Area Accelerator for Elliptic-Curve Point Multiplication on FPGA

This article presents a throughput/area accelerator for elliptic-curve point multiplication over GF(2571). To optimize the throughput, we proposed an efficient hardware accelerator architecture for a fully recursive Karatsuba multiplier to perform polynomial multiplications in one clock cycle. To minimize the hardware resources, we have utilized the proposed Karatsuba multiplier for modular square implementations. Moreover, the Itoh-Tsujii algorithm for modular inverse computation is operated using multiplier resources. These strategies permit us to reduce the hardware resources of our implemented accelerator over a large field size of 571 bits. A controller is implemented to provide control functionalities. Our throughput/area accelerator is implemented in Verilog HDL using the Vivado IDE tool. The results after the place-and-route are given on Xilinx Virtex-6 and Virtex-7 devices. The utilized slices on Virtex-6 and Virtex-7 devices are 6107 and 5683, respectively. For the same FPGA devices, our accelerator can operate at a maximum of 319 MHz and 361 MHz. The latency values for Virtex-6 and Virtex-7 devices are 28.73 μs and 25.38 μs. The comparison to the state-of-the-art shows that the proposed architecture outperforms in throughput/area values. Thus, our accelerator architecture is suitable for cryptographic applications that demand a throughput and area simultaneously.