Convolutional Neural Network Accelerator Research Articles

Design and research of high-performance convolutional neural network accelerator based on Chipyard

Abstract Neural network accelerator performs well in the research and verification of neural network models. In this paper, a convolutional neural network accelerator system composed of RISC-V processor core and Gemmini array accelerator is designed in Chisel language within the Chipyard framework, and the acceleration effect of different Gemmini array configurations for different input matrices is further investigated. The result shows that the accelerator system can achieve thousands of times acceleration compared with a single processor for large matrix calculations.

Revealing CNN Architectures via Side-Channel Analysis in Dataflow-based Inference Accelerators

Convolutional Neural Networks (CNNs) are widely used in various domains, including image recognition, medical diagnosis and autonomous driving. Recent advances in dataflow-based CNN accelerators have enabled CNN inference in resource-constrained edge devices. These dataflow accelerators utilize inherent data reuse of convolution layers to process CNN models efficiently. Concealing the architecture of CNN models is critical for privacy and security. This paper evaluates memory-based side-channel information to recover CNN architectures from dataflow-based CNN inference accelerators. The proposed attack exploits spatial and temporal data reuse of the dataflow mapping on CNN accelerators and architectural hints to recover the structure of CNN models. Experimental results demonstrate that our proposed side-channel attack can recover the structures of popular CNN models, namely Lenet, Alexnet, VGGnet16, and YOLOv2.

A Novel Two-Level Protection Scheme against Hardware Trojans on a Reconfigurable CNN Accelerator

With the boom in artificial intelligence (AI), numerous reconfigurable convolution neural network (CNN) accelerators have emerged within both industry and academia, aiming to enhance AI computing capabilities. However, this rapid landscape has also witnessed a rise in hardware Trojan attacks targeted at CNN accelerators, thereby posing substantial threats to the reliability and security of these reconfigurable systems. Despite this escalating concern, there exists a scarcity of security protection schemes explicitly tailored to counteract hardware Trojans embedded in reconfigurable CNN accelerators, and those that do exist exhibit notable deficiencies. Addressing these gaps, this paper introduces a dedicated security scheme designed to mitigate the vulnerabilities associated with hardware Trojans implanted in reconfigurable CNN accelerators. The proposed security protection scheme operates at two distinct levels: the first level is geared towards preventing the triggering of the hardware Trojan, while the second level focuses on detecting the presence of a hardware Trojan post-triggering and subsequently neutralizing its potential harm. Through experimental evaluation, our results demonstrate that this two-level protection scheme is capable of mitigating at least 99.88% of the harm cause by three different types of hardware Trojan (i.e., Trojan within RI, MAC and ReLU) within reconfigurable CNN accelerators. Furthermore, this scheme can prevent hardware Trojans from triggering whose trigger signal is derived from a processing element (PE). Notably, the proposed scheme is implemented and validated on a Xilinx Zynq XC7Z100 platform.

A Post-Quantum Encryption Mechanism Based on Convolutional Neural Network Accelerator

A Post-Quantum Encryption Mechanism Based on Convolutional Neural Network Accelerator

A High-Performance and Ultra-Low-Power Accelerator Design for Advanced Deep Learning Algorithms on an FPGA

This article addresses the growing need in resource-constrained edge computing scenarios for energy-efficient convolutional neural network (CNN) accelerators on mobile Field-Programmable Gate Array (FPGA) systems. In particular, we concentrate on register transfer level (RTL) design flow optimization to improve programming speed and power efficiency. We present a re-configurable accelerator design optimized for CNN-based object-detection applications, especially suitable for mobile FPGA platforms like the Xilinx PYNQ-Z2. By not only optimizing the MAC module using Enhanced clock gating (ECG), the accelerator can also use low-power techniques such as Local explicit clock gating (LECG) and Local explicit clock enable (LECE) in memory modules to efficiently minimize data access and memory utilization. The evaluation using ResNet-20 trained on the CIFAR-10 dataset demonstrated significant improvements in power efficiency consumption (up to 22%) and performance. The findings highlight the importance of using different optimization techniques across multiple hardware modules to achieve better results in real-world applications.

CoMN: Algorithm-Hardware Co-Design Platform for Nonvolatile Memory-Based Convolutional Neural Network Accelerators

CoMN: Algorithm-Hardware Co-Design Platform for Nonvolatile Memory-Based Convolutional Neural Network Accelerators

High Performance CNN Accelerators based on Hardware and Algorithm Co-Optimization

The efficacy of convolutional neural networks (CNNs) has led to their extensive application in picture recognition and categorization. Nevertheless, embedded systems face challenges when it comes to storing the massive volumes of weight data required by CNNs in their on-chip memory. Pruning a convolutional neural network (CNN) model can reduce its size with no impact on accuracy; however, when used with a parallel architecture, the resulting model will be slower to execute. A CNN compression method that is hardware-centric is introduced in this paper. A model of a deep neural network (DNN) will include "pruning layers (P-layers)" and "no-pruning layers (NP-layers)" attached to it. For effective and parallel processing, an NP-layer employs a regular distribution for its weights. Pruning causes a P-layer to have an irregular form, yet the high compression ratio it yields makes the shape undesirable. Using uniform and gradual quantization methods, we can achieve a processing efficiency/compression ratio trade-off with a small loss of precision. Integrating multiple parallel finite impulse response (FIR) filters into an NP-layer distributed convolutional architecture, we further improve the regular model. The P-layer irregular sparse model is suggested to use an ADF-based processing element. The suggested compression strategy and hardware architecture can be utilised by a hardware/algorithm co-optimization (HACO) method to run an NP→P hybrid compressed CNN model on FPGAs. The VGG-16 network achieves a compression ratio of 27.5× with a top-5 accuracy loss of 0.44% by using a hardware accelerator on a single FPGA chip without off-chip memory. A result of 83.0 FPS, or 1.8 times quicker, was achieved when the compressed VGG-16 model was implemented on a Xilinx VCU118 evaluation board for image applications. Index Terms - CNN, Accelerators, Co-Optimization.

Advancing Deep Learning: A Comprehensive Comparative

This manuscript conducts an in-depth comparative analysis of three seminal works in the realm of Convolutional Neural Network (CNN) accelerators using Field-Programmable Gate Arrays (FPGAs). It meticulously evaluates the architectural designs, performance benchmarks, and innovative methodologies employed in each study. The analysis delves into the intricacies of these works, spotlighting their distinct and innovative contributions to the advancement of FPGA-based CNN acceleration. By critically appraising the strengths and limitations of each approach, this paper synthesizes key insights, shedding light on prevalent trends and identifying persistent challenges within this rapidly evolving field. It discusses how each work navigates the trade-offs between efficiency, flexibility, and computational power, which are inherent in FPGA-based systems. Moreover, this comparative study serves as a comprehensive resource that encapsulates the state-of-the-art in FPGA-accelerated CNNs. It not only benchmarks the current technological landscape but also charts potential pathways for future research endeavors. By amalgamating diverse perspectives and methodologies from leading research, the paper aims to inspire and guide ongoing and future investigations, driving innovation in the optimization and application of FPGA-based CNN accelerators. This endeavor is particularly timely, given the escalating demand for high-performance, energy-efficient computational models in domains ranging from autonomous systems to real-time data analytics.

Comparative study: AutoDPR-SEM for enhancing CNN reliability in SRAM-based FPGAs through autonomous reconfiguration

Convolutional neural networks (CNNs) are widely adopted in safety-critical systems, including space applications and autonomous vehicles. Field-programmable gate arrays (FPGAs) based on SRAM are preferred for accelerating CNN computations due to their unique characteristics. However, the configuration memory of FPGAs is susceptible to single event effects (SEEs), which can corrupt computations and lead to misclassification of CNN outputs. In this study, we investigated the impact of SEEs on SRAM-based FPGAs with Two-Photon Absorption (TPA) laser fault injections through a comparative analysis of two popular CNN acceleration architectures: streaming architecture (SA) and single compute engine (SCE). Experimental results show that SA-based CNNs require more hardware resources but exhibit superior resilience against single event upsets (SEUs). Without any Radiation Hardened by Design (RHBD) protection, SCE has an error rate approximately twice as high as SA. To mitigate errors, the Xilinx IP core - Soft Error Mitigation (SEM) is used for error detection and correction, leading to error rate reductions of up to 50 % in both architectures. Importantly, we propose the AutoDPR-SEM (Autonomous Dynamic Partial Reconfiguration for Soft Error Mitigation) approach, which automatically reconfigures the SEM IP core when it remains idle due to uncorrectable errors. AutoDPR-SEM significantly improves CNN error rates, reducing errors by approximately 17.8 times in SCE and 14.8 times in SA. We also applied software level simulation to validate the TPA experiment, showing similar trends of the testing results across all models. In conclusion, the study confirms the feasibility of AutoDPR-SEM in both architectures, showcasing its potential to improve CNN error rates in safety-critical systems.

ADS-CNN: Adaptive Dataflow Scheduling for lightweight CNN accelerator on FPGAs

Lightweight convolutional neural networks (CNNs) enable lower inference latency and data traffic, facilitating deployment on resource-constrained edge devices such as field-programmable gate arrays (FPGAs). However, CNNs inference requires access to off-chip synchronous dynamic random-access memory (SDRAM), which significantly degrades inference speed and system power efficiency. In this paper, we propose an adaptive dataflow scheduling method for lightweight CNN accelerator on FPGAs named ADS-CNN. The key idea of ADS-CNN is to efficiently utilize on-chip resources and reduce the amount of SDRAM access. To achieve the reuse of logical resources, we design a time division multiplexing calculation engine to be integrated in ADS-CNN. We implement a configurable module for the convolution controller to adapt to the data reuse of different convolution layers, thus reducing the off-chip access. Furthermore, we exploit on-chip memory blocks as buffers based on the configuration of different layers in lightweight CNNs. On the resource-constrained Intel CycloneV SoC 5CSEBA6 FPGA platform, we evaluated six common lightweight CNN models to demonstrate the performance advantages of ADS-CNN. The evaluation results indicate that, compared with accelerators that use traditional tiling strategy dataflow, our ADS-CNN can achieve up to 1.29× speedup with the overall dataflow scale compression of 23.7%.

High-Speed CNN Accelerator SoC Design Based on a Flexible Diagonal Cyclic Array

The latest convolutional neural network (CNN) models for object detection include complex layered connections to process inference data. Each layer utilizes different types of kernel modes, so the hardware needs to support all kernel modes at an optimized speed. In this paper, we propose a high-speed and optimized CNN accelerator with flexible diagonal cyclic arrays (FDCA) that supports the acceleration of CNN networks with various kernel sizes and significantly reduces the time required for inference processing. The accelerator uses four FDCAs to simultaneously calculate 16 input channels and 8 output channels. Each FDCA features a 4 × 8 systolic array that contains a 3 × 3 processing element (PE) array and is designed to handle the most commonly used kernel sizes. To evaluate the proposed CNN accelerator, we mapped the widely used YOLOv5 CNN model and evaluated the performance of its implementation on the Zynq UltraScale+ MPSoC ZCU102 FPGA. The design consumes 249,357 logic cells, 2304 DSP blocks, and only 567 KB BRAM. In our evaluation, the YOLOv5n model achieves an accuracy of 43.1% (mAP@0.5). A prototype accelerator has been implemented using Samsung’s 14 nm CMOS technology. It achieves 1.075 TOPS, a peak performance with a 400 MHz clock frequency.

Improving the computational efficiency and flexibility of FPGA-based CNN accelerator through loop optimization

The convolution operation consists of three-dimensional multiply-accumulate (MAC) operations within four loops, leading to a large design space to be optimized. However, prior research did not thoroughly investigate the loop optimization operations, which led to the development of accelerators that employed inefficient parallel computing architectures and hence consumed unnecessary resources. This study addresses the limitations of existing FPGA-based Convolutional Neural Network (CNN) accelerators in terms of computational efficiency and flexibility by proposing a novel scalable accelerator architecture. We first define a design space that includes loop optimization operations such as loop tiling, loop interchange, and loop unrolling. Based on this, we explore a more efficient dataflow and accelerator architecture through a quantitative analysis of the trade-off between accelerator performance and hardware costs. Then, this paper demonstrates exploring the optimal loop optimization strategy within the design space to guide the design of accelerator architectures, advancing towards the optimal solutions for accelerator performance. The effectiveness of the suggested acceleration architecture is confirmed by implementing VGG-16, ResNet-50, and ResNet-152 on Xilinx ZCU102 and Xilinx ZCU111 FPGAs. The achieved peak throughputs for the networks are 721.48 GOPS, 546.98 GOPS, and 664.66 GOPS, demonstrating outstanding performance, efficient resource usage, and flexibility.

Implementation of FPGA-based Accelerator for Convolutional Neural Networks

This research paper presents a novel FPGA-based accelerator tailored for Convolutional Neural Networks (CNNs), specifically implemented on the Virtex-7 evaluation kit. By harnessing the inherent parallel processing capabilities of FPGAs, the architecture of the accelerator is meticulously crafted using Verilog. The FPGA implementation demonstrates a resource-efficient design, making use of 588 Look-Up Tables (LUTs) and 353 Flip Flops. Notably, the efficient utilization of these resources signifies a careful balance between computational efficiency and the available FPGA resources. This research significantly contributes to the field of hardware acceleration for CNNs by offering an optimized solution for high-performance deep learning applications. The presented architecture serves as a promising foundation for future advancements in FPGA-based accelerators, providing valuable insights for researchers and engineers working in the domain of hardware optimization for Convolutional Neural Networks.

Flare: An FPGA-Based Full Precision Low Power CNN Accelerator with Reconfigurable Structure.

Convolutional neural networks (CNNs) have significantly advanced various fields; however, their computational demands and power consumption have escalated, posing challenges for deployment in low-power scenarios. To address this issue and facilitate the application of CNNs in power constrained environments, the development of dedicated CNN accelerators is crucial. Prior research has predominantly concentrated on developing low precision CNN accelerators using code generated from high-level synthesis (HLS) tools. Unfortunately, these approaches often fail to efficiently utilize the computational resources of field-programmable gate arrays (FPGAs) and do not extend well to full precision scenarios. To overcome these limitations, we integrate vector dot products to unify the convolution and fully connected layers. By treating the row vector of input feature maps as the fundamental processing unit, we balance processing latency and resource consumption while eliminating data rearrangement time. Furthermore, an accurate design space exploration (DSE) model is established to identify the optimal design points for each CNN layer, and dynamic partial reconfiguration is employed to maximize each layer's access to computational resources. Our approach is validated through the implementation of AlexNet and VGG16 on 7A100T and ZU15EG platforms, respectively. We achieve an average convolutional layer throughput of 28.985 GOP/s and 246.711 GOP/s for full precision. Notably, the proposed accelerator demonstrates remarkable power efficiency, with a maximum improvement of 23.989 and 15.376 times compared to current state-of-the-art FPGA implementations.

Survey of convolutional neural network accelerators on field-programmable gate array platforms: architectures and optimization techniques

Survey of convolutional neural network accelerators on field-programmable gate array platforms: architectures and optimization techniques

Systematic analysis of FPGA-based hardware accelerators for convolutional neural networks

In the modern era, machine learning stands as a pivotal component of artificial intelligence, exerting a profound impact on various domains. This article delineates a methodology for designing and applying Field Programmable Gate Array (FPGA) based hardware accelerators for convolutional neural networks (CNNs). Initially, this paper introduces CNNs, a subset of deep learning techniques, and underscore their pivotal role in artificial intelligence, spanning domains such as image recognition, speech processing, and natural language understanding. Subsequently, we delve into the intricacies of FPGA, an adaptable logic device characterized by high integration and versatility, elucidating our approach to creating a hardware accelerator tailored for CNNs on the FPGA platform. To enhance computational efficiency, we employ technical strategies like dual cache structures, loop unrolling, and loop tiling for accelerating the convolutional layers. Finally, through empirical experiments employing YOLOv2, and validate the efficacy and superiority of our designed hardware accelerator model. This paper anticipates that in the forthcoming years, the methodology and research into FPGA-based CNN hardware accelerators will yield even more substantial contributions, propelling the advancement and widespread adoption of deep learning technology.

FPGA acceleration of convolutional neural networks: Current applications and future prospects

In the contemporary era of big data, the utilization of deep learning technology has become widespread in the extraction of valuable analytical insights from the data under consideration. This technology finds extensive applications in various domains, including image information recognition, speech processing, and text language analysis. When it comes to accelerating convolutional neural networks (CNNs), the utilization of Field-Programmable Gate Arrays (FPGAs) offers distinct advantages over other hardware accelerators. Nevertheless, it is important to acknowledge that FPGA-based acceleration also comes with its own structural limitations. This article places its focus on two main aspects: firstly, it delves into the present-day application landscape and the latest developmental trajectories of convolutional neural networks. Secondly, it elucidates the inherent characteristics of FPGA implementations of CNNs. Furthermore, the article conducts a comprehensive examination of the pertinent constraints associated with FPGA-accelerated deep learning algorithms. The discussion extends beyond the present moment and ventures into the future, offering insights into the potential advancements in the field of deep learning. Importantly, it also brings into focus the prospective avenues for further research regarding the application of FPGAs in the domain of convolutional neural networks.

REC: REtime Convolutional layers to fully exploit harvested energy for ReRAM-based CNN accelerators

As the Internet of Things (IoTs) increasingly combines AI technology, it is a trend to deploy neural network algorithms at edges and make IoT devices more intelligent than ever. Moreover, energy-harvesting technology-based IoT devices have shown the advantages of green and low-carbon economy, convenient maintenance, and theoretically infinite lifetime, etc. However, the harvested energy is often unstable, resulting in low performance due to the fact that a fixed load cannot sufficiently utilize the harvested energy. To address this problem, recent works focusing on ReRAM-based convolutional neural networks (CNN) accelerators under harvested energy have proposed hardware/software optimizations. However, those works have overlooked the mismatch between the power requirement of different CNN layers and the variation of harvested power. Motivated by the above observation, this paper proposes a novel strategy, called REC , that retimes convolutional layers of CNN inferences to improve the performance and energy efficiency of energy harvesting ReRAM-based accelerators. Specifically, at the offline stage, REC defines different power levels to fit the power requirements of different convolutional layers. At runtime, instead of sequentially executing the convolutional layers of an inference one by one, REC retimes the execution timeframe of different convolutional layers so as to accommodate different CNN layers to the changing power inputs. What is more, REC provides a parallel strategy to fully utilize very high power inputs. Moreover, a case study is presented to show that REC is effective to improve the real-time accomplishment of periodical critical inferences because REC provides an opportunity for critical inferences to preempt the process window with a high power supply. Our experimental results show that the proposed REC scheme achieves an average performance improvement of 6.1 × (up to 16.5 ×) compared to the traditional strategy without the REC idea. The case study results show that the REC scheme can significantly improve the success rate of periodical critical inferences’ real-time accomplishment.

XVDPU: A High-Performance CNN Accelerator on the Versal Platform Powered by the AI Engine

Today, convolutional neural networks (CNNs) are widely used in computer vision applications. However, the trends of higher accuracy and higher resolution generate larger networks. The requirements of computation or I/O are the key bottlenecks. In this article, we propose XVDPU: the AI Engine (AIE)-based CNN accelerator on Versal chips to meet heavy computation requirements. To resolve the IO bottleneck, we adopt several techniques to improve data reuse and reduce I/O requirements. An arithmetic logic unit is further proposed that can better balance resource utilization, new feature support, and efficiency of the whole system. We have successfully deployed more than 100 CNN models with our accelerator. Our experimental results show that the 96-AIE-core implementation can achieve 1,653 frames per second (FPS) for ResNet50 on VCK190, which is 9.8× faster than the design on ZCU102 running at 168.5 FPS. The 256-AIE-core implementation can further achieve 4,050 FPS. We propose a tilling strategy to achieve feature-map-stationary for high-definition CNN with the accelerator, achieving 3.8× FPS improvement on the residual channel attention network and 3.1× on super-efficient super-resolution. This accelerator can also solve the 3D convolution task in disparity estimation, achieving end-to-end performance of 10.1 FPS with all the optimizations.

Dataflow optimization with layer-wise design variables estimation method for enflame CNN accelerators

As convolution layers have been proved to be the most time-consuming operation in convolutional neural network (CNN) algorithms, many efficient CNN accelerators have been designed to boost the performance of convolution operations. Previous works on CNN acceleration usually use fixed design variables for diverse convolutional layers, which would lead to inefficient data movements and low utilization of computing resource. We tackle this issue by proposing a flexible dataflow optimization method with design variables estimation for different layers. The optimization method first narrows the design space by the priori constraints, and then enumerates all legal solutions to select the optimal design variables. We demonstrate the effectiveness of the proposed optimization method by implementing representative CNN models (VGG-16, ResNet-18 and MobileNet V1) on Enflame Technology's programmable CNN accelerator, General Computing Unit (GCU). The results indicate that our optimization can significantly enhance the throughput of the convolution layers in ResNet, VGG and MobileNet on GCU, with improvement of up to 1.84×. Furthermore, it achieves up to 2.08× of GCU utilization specifically for the convolution layers of ResNet on GCU.