Systolic Array Research Articles

Implementation of Systolic Multiplier Using Hybrid Multiplexer Dependent Adder

Multipliers are basic building blocks in various integrated circuits like microprocessors, micro controllers, and ALUs. The existing multipliers suffer from high power consumption and inefficient use of hardware resources. They often rely on traditional adder structures that are not tailored for specific operations, leading to suboptimal performance. Additionally, their fixed architectures limit adaptability and scalability in different applications. So, the proposed approach offers enhanced computational efficiency and reduced power consumption compared to conventional multiplier designs. By integrating multiplexer-dependent adders into the systolic array, the proposed method optimizes resource utilization and delivers improved performance for various arithmetic operations. This integration allows for dynamic selection of adder types based on the specific multiplication operation, significantly reducing power consumption and latency. By adapting the hardware resources to computational needs, the method achieves higher efficiency and flexibility, making it suitable for a wide range of applications in digital signal processing and data processing systems.

Design and Implementation of an NoC-Based Convolution Architecture With GEMM and Systolic Arrays

Design and Implementation of an NoC-Based Convolution Architecture With GEMM and Systolic Arrays

Adaptive neural acceleration unit based on heterogeneous multicore hardware architecture FPGA and software-defined hardware

ABSTRACT This study is anchored in a heterogeneous multicore hardware architecture, specifically Field Programmable Gate Arrays (FPGAs), and software-defined hardware. It employs dynamic planning for the reconfiguration of a software-defined System-on-Chip (SOC) to expedite neural network processes. The reconfigured software-defined SOC serves as an efficient data processor, facilitating the hybrid development and verification of domain-specific System-on-Chip architectures, adhering to the Advanced Microcontroller Bus Architecture (AMBA) standard. The proposed block data trimming methodology significantly enhances overall hybrid computing efficiency by mitigating throughput limitations inherent in Systolic array hardware. The designed Systolic array Matrix Multiply Unit (MMU) is evaluated for a maximum MMU size of 32 × 32 and 1,024 Multiply Accumulator (MAC) units. Hybrid dynamic circuits are devised for the synthesis of int8, int16, int32, and int64 data types and associated logic circuits to validate the performance of parallel computing. It is anticipated that this proposed system can find utility in the development and deployment of unmanned devices, as well as in the integration of deep learning and multi-sensory fusion involving acoustic, tactile, and force sensory components.

Efficient FPGA Implementation of Convolutional Neural Networks and Long Short-Term Memory for Radar Emitter Signal Recognition

In recent years, radar emitter signal recognition has enjoyed a wide range of applications in electronic support measure systems and communication security. More and more deep learning algorithms have been used to improve the recognition accuracy of radar emitter signals. However, complex deep learning algorithms and data preprocessing operations have a huge demand for computing power, which cannot meet the requirements of low power consumption and high real-time processing scenarios. Therefore, many research works have remained in the experimental stage and cannot be actually implemented. To tackle this problem, this paper proposes a resource reuse computing acceleration platform based on field programmable gate arrays (FPGA), and implements a one-dimensional (1D) convolutional neural network (CNN) and long short-term memory (LSTM) neural network (NN) model for radar emitter signal recognition, directly targeting the intermediate frequency (IF) data of radar emitter signal for classification and recognition. The implementation of the 1D-CNN-LSTM neural network on FPGA is realized by multiplexing the same systolic array to accomplish the parallel acceleration of 1D convolution and matrix vector multiplication operations. We implemented our network on Xilinx XCKU040 to evaluate the effectiveness of our proposed solution. Our experiments show that the system can achieve 7.34 giga operations per second (GOPS) data throughput with only 5.022 W power consumption when the radar emitter signal recognition rate is 96.53%, which greatly improves the energy efficiency ratio and real-time performance of the radar emitter recognition system.

HSAS: Efficient task scheduling for large scale heterogeneous systolic array accelerator cluster

To efficiently process a large amount of deep neural network models can be challenging, due to significant differences among models and even layers. Nowadays, systolic array has become a common architecture for processing neural networks. With this architecture, different array sizes can lead to huge difference in hardware utilization for the same network. Therefore, to achieve the optimal processing efficiency for a large amount of models, a heterogeneous systolic array accelerator cluster could be more advantageous than a homogeneous architecture. In this work, we propose such heterogeneous architecture, and design its scheduling algorithm HSAS. HSAS can evaluate how models fit with systolic arrays, by our systolic array performance and energy models. Meanwhile, HSAS also takes load balance and preemption into consideration. We further introduce a task decomposition algorithm and subtask priority management table, to enable more fine-grained subtask level scheduling. Our evaluation shows task level HSAS can improve average normalized turnaround time, system throughput and fairness by up to more than 80% compared with classic and state-of-the-art methods, while subtask level HSAS can achieve 18%–63% improvement compared to other methods.

SparGD: A Sparse GEMM Accelerator with Dynamic Dataflow

Deep learning has become a highly popular research field, and previously deep learning algorithms ran primarily on CPUs and GPUs. However, with the rapid development of deep learning, it was discovered that existing processors could not meet the specific large-scale computing requirements of deep learning, and custom deep learning accelerators have become popular. The majority of the primary workloads in deep learning are general matrix-matrix multiplications (GEMMs), and emerging GEMMs are highly sparse and irregular. The TPU and SIGMA are typical GEMM accelerators in recent years, but the TPU does not support sparsity, and both the TPU and SIGMA have insufficient utilization rates of the Processing Element (PE). We design and implement SparGD, a sparse GEMM accelerator with dynamic dataflow. SparGD has specific PE structures, flexible distribution networks and reduction networks, and a simple dataflow switching module. When running sparse and irregular GEMMs, SparGD can maintain high PE utilization while utilizing sparsity, and can switch to the optimal dataflow according to the computing environment. For sparse, irregular GEMMs, our experimental results show that SparGD outperforms systolic arrays by 30 times and SIGMA by 3.6 times.

Accelerating Strawberry Ripeness Classification Using a Convolution-Based Feature Extractor along with an Edge AI Processor

Image analysis-based artificial intelligence (AI) models leveraging convolutional neural networks (CNN) take a significant role in evaluating the ripeness of strawberry, contributing to the maximization of productivity. However, the convolution, which constitutes the majority of the CNN models, imposes significant computational burdens. Additionally, the dense operations in the fully connected (FC) layer necessitate a vast number of parameters and entail extensive external memory access. Therefore, reducing the computational burden of convolution operations and alleviating memory overhead is essential in embedded environment. In this paper, we propose a strawberry ripeness classification system utilizing a convolution-based feature extractor (CoFEx) for accelerating convolution operations and an edge AI processor, Intellino, for replacing FC layer operations. We accelerated feature map extraction utilizing the CoFEx constructed with systolic array (SA) and alleviated the computational burden and memory overhead associated with the FC layer operations by replacing them with the k-nearest neighbors (k-NN) algorithm. The CoFEx and the Intellino both were designed with Verilog HDL and implemented on a field-programmable gate array (FPGA). The proposed system achieved a high precision of 93.4%, recall of 93.3%, and F1 score of 0.933. Therefore, we demonstrated a feasibility of the strawberry ripeness classification system operating in an embedded environment.

Optimized systolic array filters with noise classification for extracting FECG

To preserve the lives of both mother and foetus during the early phases of childbirth, heart abnormality diagnosis is essential. In remote places where understanding of maternal care is limited, the death rate caused by carelessness or a failure to detect abnormalities is still a problem. The signal will contain various external or internal noise sources to isolate the key components of the foetal heart rate from the mother's belly during labour. The likelihood that the genuine signal would be misinterpreted and result in a false report will increase in the presence of such noise sources. Although there are many software programs available for extracting the QRS features from foetal ECG signals, it is unavoidable that specialized hardware is required for a significant reduction in both area and power. This paper's main goal is to extract the QRS complex using LDA, then improve Social Spider classifier performance using the suggested TAODV as a distance metric calculator, and then compare against existing methods to discover sounds that are distorting the normal heart rate. Systolic array filter with suggested Glitch Avoidance Circuit employing MUX is simulated using Cadence Virtuoso in 65nm technology to remove noise from the observed QRS complex. Over 100 records with the necessary examples from MIT-BIH Arrhythmia were used in the simulations, and it was discovered that MATLAB 2010b was used to adopt a unique technique for classifying noise. The suggested TAODV-based SSA classifier's accuracy is 96.8%, whereas the accuracy of a filter with a glitch avoidance circuit is 96.13%. The primary benefit of these strategies comprises cutting-edge hardware and computational solutions.

Computing Acceleration to Genome-Wide Association Study Based on CPU/FPGA Heterogeneous System

Genome Wide Association Study (GWAS) reveals the influence of single nucleotide polymorphisms (SNP) and other genetic markers on the complex genetic disease traits, making a significant contribution to the prevention and treatment of genetic diseases. Since GWAS needs to calculate large numbers of pairwise tests, it is highly time-consuming and this limits its widespread promotion in many medical applications. To solve the above problem, many studies deploy GWAS on the CPU/GPU platform. By taking advantages of GPUs' high parallelism, the completion time of GWAS can be significantly decreased. However, GPUs have high energy consumption, which increases GWAS's operating cost. Considering FPGA has low power, high parallelism and programmability, this article aims to use FPGA to design a hardware accelerator dedicated to GWAS for the purpose of optimizing both completion time and energy cost of GWAS. First, the CPU/FPGA heterogenous computing system is built, and the computationally-intensive GWAS tasks such as the contingency table creation (CTC) and Kirkwood Superposition Approximation (KSA) screening are offloaded to run on FPGA. Then, a specific FPGA architectures dedicated to CTC and KSA tasks of GWAS are implemented by using a set of architectural design technologies such as hardware and software co-optimization, systolic array, loop unrolling, pipeline and parallelism. To evaluate the performance of FPGA, we compare its completion time, energy cost and performance per Watt with those of CPU and GPU. The results show the FPGA and GPU can respectively achieve 105--fold and 148--fold speedup compared with CPU. Although FPGA's computing performance is a bit lower than that of GPU, it has higher energy efficiency and its performance per Watt is 1.4 times that of GPU.

TREAD-M3D: Temperature-Aware DNN Accelerators for Monolithic 3-D Mobile Systems

Monolithic 3D (MONO3D) integration provides performance and power efficiency benefits over 2D circuits and, thus, is a potent technology for the design of Deep Neural Network (DNN) accelerators with enhanced energy efficiency. However, high IC temperatures are major challenges for the design of MONO3D systems. To this end, this paper focuses on designing temperature-aware MONO3D DNN accelerators. We propose a new automated method, called TREAD-M3D, that provides a near-optimal MONO3D DNN accelerator architecture in terms of systolic array size, SRAM organization, partition across 3D layers, and operating frequency, for a given DNN, optimization goal, and temperature constraint. TREAD-M3D incorporates circuit-and architecture-level models to evaluate the power and performance characteristics of different partitions. Our method reveals valuable insights and enables tradeoff analysis for achieving high energy efficiency in MONO3D systolic arrays. In comparison to recent works that adopt a fixed partition choice to design MONO3D DNN systems, TREAD-M3D yields up to 22% higher energy efficiency. Using TREAD-M3D, we further demonstrate that temperature unawareness not only leads to infeasible configurations due to temperature violations but also over-estimates energy-delay-product benefits by up to 24%.

Accelerating Attention Mechanism on FPGAs based on Efficient Reconfigurable Systolic Array

Transformer model architectures have recently received great interest in natural language, machine translation, and computer vision, where attention mechanisms are their building blocks. However, the attention mechanism is expensive because of its intensive matrix computations and complicated data flow. The existing hardware architecture has some disadvantages for the computing structure of attention, such as inflexibility and low efficiency. Most of the existing papers accelerate attention by reducing the amount of computation through various pruning algorithms, which will affect the results to a certain extent with different sparsity. This paper proposes the hardware accelerator for the multi-head attention (MHA) on field-programmable gate arrays (FPGAs) with reconfigurable architecture, efficient systolic array, and hardware-friendly radix-2 softmax. We propose a novel method called Four inputs Processing Element (FPE) to double the computation rate of the data-aware systolic array (SA) and make it efficient and load balance. Especially, the computation framework is well designed to ensure the utilization of SA efficiently. Our design is evaluated on a Xilinx Alveo U250 card, and the proposed architecture achieves 51.3×, 17.3× improvement in latency, and 54.4×, 17.9× energy savings compared to CPU and GPU.

A new design approach of hardware implementation through natural language entry

AbstractOpenAI's ChatGPT (GPT‐4) ushers in a superior mode of computer interaction through natural language dialogues. Notably, it generates not only engaging dialogues but also codes aligned to queries and requirements. The potential of ChatGPT in hardware implementation via natural language is implemented and a strategy for “asking the right questions” is outlined. The versatility of ChatGPT is demonstrated through three mainstream hardware designs – systolic array, ResNet and MobileNet accelerators – comparing these with hand‐coded designs. The evaluation metrics include design quality, design efforts, and limitations of code generated by ChatGPT/GPT‐4/Cursor against prevalent High‐Level Synthesis or hand‐coded HDL designs. Consequently, a novel design workflow is proposed and the constraints of using GPT, particularly in AI accelerators, are highlighted.

An Area-Efficient Unified VLSI Architecture for Type IV DCT/DST Having an Efficient Hardware Security with Low Overheads

This paper introduces an efficient solution for designing a unified VLSI implementation for type IV DCT/DST while solving one challenging problem in obtaining high performance VLSI chips for common goods, which is solving the security of the hardware while obtaining a VLSI implementation with high performance. The new solution uses a new systolic array algorithm for type IV DST that can allow us to obtain an efficient unified VLSI architecture with one previously designed for type IV DCT. The proposed method uses special arithmetic structures that have been called quasi-cycle convolutions that can be efficiently mapped on linear systolic arrays. Moreover, the obtained unified VLSI architecture, besides being an efficient implementation with a low hardware complexity and high-speed performance, allows for an efficient inclusion of the obfuscation technique with very low overheads.

Exploiting data encoding and reordering for low-power streaming in systolic arrays

Systolic Array (SA) architectures are well-suited for accelerating matrix multiplications through the use of a pipelined array of Processing Elements (PEs) communicating with local connections and pre-orchestrated data movements. Even though most of the dynamic power consumption in SAs is due to multiplications and additions, pipelined data movement within the SA constitutes an additional important contributor. The goal of this work is to reduce the dynamic power consumption associated with the feeding of data to the SA, by employing both dynamic (run-time) and static (offline) techniques. At the hardware level, the proposed architecture synergistically applies bus-invert coding and zero-value clock gating. By exploiting salient attributes of state-of-the-art CNNs, such as the value distribution of the weights, the proposed SA applies appropriate encoding only to the data that exhibits high switching activity. Similarly, when one of the inputs is zero, unnecessary operations are entirely skipped. In addition to this duet of run-time techniques, the proposed methodology also leverages the inherent property of the weight matrix to remain unchanged throughout the inference phase. As such, the weight matrix is appropriately reordered offline to minimize the switching activity between consecutive values, as the matrix is repeatedly loaded into the array. The weight reordering process is formulated as a Traveling Salesman Problem (TSP) and its solution is translated into a switching-activity-aware row permutation of the weight matrix. The symbiotic combination of selectively targeted, application-aware dynamic encoding and offline weight reordering is demonstrated to reduce the switching activity by 38%, on average. This translates to an overall dynamic power reduction of 17.1%–23% when executing state-of-the-art CNN layers on an SA of size 32 × 32. These power savings scale with the array size; for an array of size 64 × 64, the proposed design consumes 29.7%–35.4% less power.

Low-complexity systolic array structure for field multiplication in resource-constrained IoT nodes

Low-complexity systolic array structure for field multiplication in resource-constrained IoT nodes

A configurable hardware-efficient ECG classification inference engine based on CNN for mobile healthcare applications

—Electrocardiogram (ECG) processors for healthcare have been widely used, however most of them can only adapt to specific applications, lacking flexibility. For achieving scalable on-chip ECG classification, a flexible inference engine based on one-dimensional (1-D) convolutional neural network (CNN) is proposed. By utilizing the proposed computing strategy based on systolic arrays, filter level parallelism and output channel parallelism are achieved. The configurable processing units (PUs) and modifiable instruction registers allow this inference engine to support computing of 1-D convolutional layers or fully connected layers with different scales. Based on the proposed data buffers system with multi-level storage structure, the input feature values and weight values are reused, thereby reducing hardware overhead and memory access power. This design has been validated on FPGA using our proposed two arrhythmia classification models that represent low energy consumption and high accuracy requirements, achieving accuracy of 98.9% and 99.3%, respectively, with energy consumption of 2.05 μJ and 14.27 μJ per classification at 200 MHz. The hardware implementation results indicate that good configurability enables our design to adapt to different ECG classification scenarios, effectively improving the hardware universality in mobile healthcare applications.

An Electro-Photonic System for Accelerating Deep Neural Networks

The number of parameters in deep neural networks (DNNs) is scaling at about 5× the rate of Moore’s Law. To sustain this growth, photonic computing is a promising avenue, as it enables higher throughput in dominant general matrix-matrix multiplication (GEMM) operations in DNNs than their electrical counterpart. However, purely photonic systems face several challenges including lack of photonic memory and accumulation of noise. In this article, we present an electro-photonic accelerator, ADEPT, which leverages a photonic computing unit for performing GEMM operations, a vectorized digital electronic application-specific integrated circuits for performing non-GEMM operations, and SRAM arrays for storing DNN parameters and activations. In contrast to prior works in photonic DNN accelerators, we adopt a system-level perspective and show that the gains while large are tempered relative to prior expectations. Our goal is to encourage architects to explore photonic technology in a more pragmatic way considering the system as a whole to understand its general applicability in accelerating today’s DNNs. Our evaluation shows that ADEPT can provide, on average, 5.73× higher throughput per watt compared to the traditional systolic arrays in a full-system, and at least 6.8× and 2.5× better throughput per watt, compared to state-of-the-art electronic and photonic accelerators, respectively.

STRAIT: Self-Test and Self-Recovery for AI Accelerator

As the demand for data-intensive analytics has increased with the rapid advance in artificial intelligence (AI), various AI accelerators have been proposed. However, as AI-based solutions have been adapted to applications requiring accuracy and reliability, the reliability of them has become a critical issue. For this reason, self-test and self-recovery for AI accelerator (STRAIT) is proposed in this paper. It facilitates self-test, self-diagnosis, and self-recovery by utilizing the structural and operational characteristics of systolic array in AI accelerator. The proposed self-test is progressed using scan chains composed of functional paths, and can achieve a 100% test coverage (for both stuck-at and transition-delay faults) with small number of test patterns and reduced test power. The proposed self-diagnosis is progressed with the proposed self-test in real-time, and allows accurate fault localization with fault type analysis. The proposed self-recovery is progressed using efficient pruning for faulty processing elements with weight allocation, and the reliability of AI accelerators can drastically increase with negligible performance degradation. However, STRAIT can be implemented with a small area overhead.

A Survey of Design and Optimization for Systolic Array-based DNN Accelerators

In recent years, it has been witnessed that the systolic array is a successful architecture for DNN hardware accelerators. However, the design of systolic arrays also encountered many challenges. As DNN structures and applications become more complex, a DNN hardware accelerator based on the typical systolic array architecture suffers severe performance and efficiency penalties. So, it has motivated a significant amount of research on the redesign and optimization of the systolic array architecture. In this article, we survey these works on analyzing, redesigning, and improving the performance and efficiency of the systolic array architecture. These works are critical to the design flow of DNN accelerators based on systolic arrays. We also provide a technique classification of these works on the basis of their main research idea. Further, we attempt to compare the advantages and disadvantages of different designs and different technologies and provide quantitative results for reference. The aim of this survey is to provide researchers with knowledge of the state-of-the-art in the systolic array architecture and motivate them to design highly efficient DNN accelerators of tomorrow.

Design and Implementation of Handwritten Digit Recognition Accelerator Based on Systolic Array

Due to the rapid development in artificial intelligence, the classic Von Neumann architecture is no longer able to meet the demands of high-computing, high-storage, and high-bandwidth artificial intelligence applications. To address this issue, this paper proposes a handwritten digit recognition accelerator based on a systolic array. First, a 5 convolutional layer handwritten digit recognition network is built. Besides, The FPGA-based convolution accelerator employs im2col technology to convert convolution calculations into matrix multiplications and uses a systolic array to efficiently perform these multiplications. Furthermore, the four convolution processes are pipelined to improve the throughput. Finally, a handwritten digit recognition system is implemented on the PYNQ-Z2. Compared with the software implementation on Arm, the speedup of the accelerator is 598x, the delay is 1.24 s, and the power consumption is 2.51 w.