Neural Network Accelerator Research Articles

Photonic diffractive generators through sampling noises from scattering media

Photonic computing, with potentials of high parallelism, low latency and high energy efficiency, have gained progressive interest at the forefront of neural network (NN) accelerators. However, most existing photonic computing accelerators concentrate on discriminative NNs. Large-scale generative photonic computing machines remain largely unexplored, partly due to poor data accessibility, accuracy and hardware feasibility. Here, we harness random light scattering in disordered media as a native noise source and leverage large-scale diffractive optical computing to generate images from above noise, thereby achieving hardware consistency by solely pursuing the spatial parallelism of light. To realize experimental data accessibility, we design two encoding strategies between images and optical noise latent space that effectively solves the training problem. Furthermore, we utilize advanced photonic NN architectures including cascaded and parallel configurations of diffraction layers to enhance the image generation performance. Our results show that the photonic generator is capable of producing clear and meaningful synthesized images across several standard public datasets. As a photonic generative machine, this work makes an important contribution to photonic computing and paves the way for more sophisticated applications such as real world data augmentation and multi modal generation.

DSA-CNN: an fpga-integrated deformable systolic array for convolutional neural network acceleration

DSA-CNN: an fpga-integrated deformable systolic array for convolutional neural network acceleration

High-speed programming with threshold division for RRAM-based neural network accelerators

High-speed programming with threshold division for RRAM-based neural network accelerators

EncodingNet: A Novel Encoding-Based MAC Design for Efficient Neural Network Acceleration

EncodingNet: A Novel Encoding-Based MAC Design for Efficient Neural Network Acceleration

SENNA: Unified Hardware/Software Space Exploration for Parametrizable Neural Network Accelerators

Parametrizable neural network accelerators enable the deployment of targeted hardware for specialized environments. Finding the best architecture configuration for a given specification, however, is challenging. A large number of hardware configurations have to be considered, and for each hardware instance, an efficient software execution plan needs to be found, leading to a vast search space. Prior work has tackled this problem by dividing the search into subproblems for individual layers of a network. There is no guarantee, however, that the overall best hardware configuration that delivers the desired end-to-end performance across the entire network is among the best individual layer configurations. This work presents SENNA, a unified hardware/software space exploration framework for parametrizable neural network accelerators. To guide the exploration towards the overall best configuration, SENNA employs a multi-objective genetic algorithm with a novel design space representation that encodes the configuration of hardware and software parameters in a single chromosome. Using the Parallel Island Model (PIM), each layer is represented by one or more individual islands each containing a separate population to simultaneously search for the best configuration across the entire network. A tailored gene migration technique enables the exchange of genes between the populations of different islands. SENNA is evaluated with three parametrizable architectures and four neural networks. The evaluation result demonstrates that SENNA achieves upto 1.92x EDP improvement compared to the State-of-the-Art. With equivalent evaluation budgets, SENNA shows 2.5x-9.3x speedup compared to an Oracle scheme and the State-of-the-Art.

Filter pruning via annealing decaying for deep convolutional neural networks acceleration

Filter pruning via annealing decaying for deep convolutional neural networks acceleration

Enabling On-Device Inference of Large Language Models : Challenges, Techniques, and Applications

This comprehensive article explores the cutting-edge techniques and challenges associated with on-device inference of Large Language Models (LLMs), a transformative approach that brings advanced AI capabilities directly to mobile and edge devices. The article delves into the intricate balance between the computational demands of LLMs and the resource constraints of mobile hardware, presenting a detailed analysis of various strategies to overcome these limitations. Key areas of focus include model compression techniques such as pruning and knowledge distillation, quantization methods, and the development of efficient model architectures. The article also examines the role of specialized hardware accelerators, including Neural Processing Units (NPUs), FPGAs, and ASICs, in enhancing on-device performance. Additionally, the article addresses critical aspects of memory management and optimization strategies crucial for efficient LLM deployment. Through a rigorous evaluation of performance metrics, the article offers insights into the trade-offs between model size, inference speed, and accuracy. It further explores diverse applications and use cases, from real-time language translation to privacy-preserving text analysis, highlighting the transformative potential of on-device LLM inference. The article concludes with an examination of ongoing challenges and future research directions, including improving energy efficiency, enhancing model adaptability, and addressing privacy and security concerns. This comprehensive article provides researchers, developers, and industry professionals with a thorough understanding of the current state and future prospects of on-device LLM inference, underlining its significance in shaping the next generation of AI-powered mobile and IoT applications.

A comprehensive survey on graph neural network accelerators

A comprehensive survey on graph neural network accelerators

Reconfigurable Acceleration of Neural Networks: A Comprehensive Study of FPGA-based Systems

This paper explores the potential of Field-Programmable Gate Arrays (FPGAs) for accelerating both neural network inference and training. We present a comprehensive analysis of FPGA-based systems, encompassing architecture design, hardware implementation strategies, and performance evaluation. Our study highlights the advantages of FPGAs over traditional CPUs and GPUs for neural network workloads, including their inherent parallelism, reconfigurability, and ability to tailor hardware to specific network needs. We delve into various hardware implementation strategies, from direct mapping to dataflow architectures and specialized hardware blocks, examining their impact on performance. Furthermore, we benchmark FPGA-based systems against traditional platforms, evaluating inference speed, energy efficiency, and memory bandwidth. Finally, we explore emerging trends in FPGA-based neural network acceleration, such as specialized architectures, efficient memory management techniques, and hybrid CPU-FPGA systems. Our analysis underscores the significant potential of FPGAs for accelerating deep learning applications, particularly those requiring high performance, low latency, and energy efficiency.

An energy-efficient dehazing neural network accelerator based on E$$^2$$AOD-Net

An energy-efficient dehazing neural network accelerator based on E$$^2$$AOD-Net

An Efficient FPGA-Based Dilated and Transposed Convolutional Neural Network Accelerator

An Efficient FPGA-Based Dilated and Transposed Convolutional Neural Network Accelerator

A Cascaded ReRAM-based Crossbar Architecture for Transformer Neural Network Acceleration

Emerging resistive random-access memory (ReRAM) based processing-in-memory (PIM) accelerators have been increasingly explored in recent years because they can efficiently perform in-situ matrix-vector multiplication (MVM) operations involved in a wide spectrum of artificial neural networks. However, there remain significant challenges to apply existing ReRAM-based PIM accelerators to the most popular Transformer neural networks. Since Transformers involve a series of matrix-matrix multiplication (MatMul) operations with data dependencies, they should write intermediate results of MatMuls to ReRAM crossbar arrays for further processing. Conventional ReRAM-based PIM accelerators often suffer from high latency of ReRAM writes and intra-layer pipeline stalls. In this paper, we propose ReCAT, a ReRAM-based PIM accelerator designed particularly for Transformers. ReCAT exploits transimpedance amplifiers (TIAs) to cascade a pair of crossbar arrays for MatMul operations involved in the self-attention mechanism. The intermediate result of a MatMul generated by one crossbar array can be directly mapped to another crossbar array, avoiding costly analog-to-digital conversions. In this way, ReCAT allows MVM operations to overlap with the corresponding data mapping, hiding the high latency of ReRAM writes. Furthermore, we propose an analog-to-digital converter (ADC) virtualization scheme to dynamically share scarce ADCs among a group of crossbar arrays, and thus significantly improve the utilization of ADCs to eliminate the performance bottleneck of MVM operations. Experimental results show that ReCAT achieves 207.3 ×, 2.11 ×, and 3.06 × performance improvement on average compared with other Transformer acceleration solutions—GPUs, ReBert, and ReTransformer, respectively.

LDF-BNN: A Real-Time and High-Accuracy Binary Neural Network Accelerator Based on the Improved BNext.

Significant progress has been made in industrial defect detection due to the powerful feature extraction capabilities of deep neural networks (DNNs). However, the high computational cost and memory requirement of DNNs pose a great challenge to the deployment of industrial edge-side devices. Although traditional binary neural networks (BNNs) have the advantages of small storage space requirements, high parallel computing capability, and low power consumption, the problem of significant accuracy degradation cannot be ignored. To tackle these challenges, this paper constructs a BNN with layered data fusion mechanism (LDF-BNN) based on BNext. By introducing the above mechanism, it strives to minimize the bandwidth pressure while reducing the loss of accuracy. Furthermore, we have designed an efficient hardware accelerator architecture based on this mechanism, enhancing the performance of high-accuracy BNN models with complex network structures. Additionally, the introduction of multi-storage parallelism alleviates the limitations imposed by the internal transfer rate, thus improving the overall computational efficiency. The experimental results show that our proposed LDF-BNN outperforms other methods in the comprehensive comparison, achieving a high accuracy of 72.23%, an image processing rate of 72.6 frames per second (FPS), and 1826 giga operations per second (GOPs) on the ImageNet dataset. Meanwhile, LDF-BNN can also be well applied to defect detection dataset Mixed WM-38, achieving a high accuracy of 98.70%.

Realizing In-Memory Computing using Reliable Differential 8T SRAM for Improved Latency

Traditional von Neumann computing architectures suffer from high energy and lower speed as compared to the requirements of modern applications like those required in neural network accelerators. A modified differential eight transistor (8 + T) static random access memory (SRAM)-based in-memory computing (IMC) structure was presented for realizing bit-wise Boolean logic operations. The 8 + T SRAM-IMC is designed at the 32 nm technology node with throughput of 2.1849, 2.4815, 2.5795, 2.6240, 2.6495, 2.6619, 2.6690, 2.6732, and 2.6749 giga outputs per second for 0.5 to 1.3 V supply voltage range, respectively. The differential 8T cell used to implement logic operations supports NAND and NOR operations with minimal overhead while also performing the standard storage operation with added stability over the conventional 6T and 8T SRAM cells. The SRAM-IMC offers reliable Boolean logic operations by using asymmetric sensing strategy for all process corners, TT, SS, SF, FS, and FF for an operating temperature range of 220 K to 400 K. Monte Carlo simulation considering global threshold voltage deviation of 50 mV was performed for various operating conditions to study the impact of variations on design parameters such as latency.

An efficient CNN accelerator for pattern-compressed sparse neural networks on FPGA

Currently, the sparsity of weights and activations are mainly utilized to improve the energy efficiency and computational performance of CNN accelerators. However, the irregular sparsity of weights and activations can lead to problems such as mapping access conflict and imbalanced workload, which poses great challenges to the efficient computation of sparse networks. To address those problems, this paper proposes a CNN accelerator for pattern-compressed sparse neural networks on FPGA to achieve efficient inference acceleration. It mainly includes three parts: first, we propose a convolutional kernel grouping fusion method to achieve a balanced workload of weights. Second, we propose an activation hash mapping conflict reduction method to drastically reduce the mapping access conflict rate. Finally, we further propose a hierarchical mapping computing architecture to achieve efficient processing of mapping access conflicts and balanced adaptation of weight loads. To verify the effectiveness of the method proposed in this paper, our accelerator is designed and implemented on the Zynq UltraScale+ MPSoC ZCU102 evaluation board and evaluated by running VGG16 and ResNet50 networks. The experimental results show that the method proposed in this paper can reduce the mapping conflicts by more than 97 % and improve the utilization rate of processing element (PE) by 1.67×. Furthermore, the accelerator implemented in this paper can achieve 393.2 GOPs throughput, 1.40–3.25× computational performance, and 1.11–7.34× energy efficiency improvement.

NEST‐C: A deep learning compiler framework for heterogeneous computing systems with artificial intelligence accelerators

AbstractDeep learning (DL) has significantly advanced artificial intelligence (AI); however, frameworks such as PyTorch, ONNX, and TensorFlow are optimized for general‐purpose GPUs, leading to inefficiencies on specialized accelerators such as neural processing units (NPUs) and processing‐in‐memory (PIM) devices. These accelerators are designed to optimize both throughput and energy efficiency but they require more tailored optimizations. To address these limitations, we propose the NEST compiler (NEST‐C), a novel DL framework that improves the deployment and performance of models across various AI accelerators. NEST‐C leverages profiling‐based quantization, dynamic graph partitioning, and multi‐level intermediate representation (IR) integration for efficient execution on diverse hardware platforms. Our results show that NEST‐C significantly enhances computational efficiency and adaptability across various AI accelerators, achieving higher throughput, lower latency, improved resource utilization, and greater model portability. These benefits contribute to more efficient DL model deployment in modern AI applications.

Design and research of high-performance convolutional neural network accelerator based on Chipyard

Abstract Neural network accelerator performs well in the research and verification of neural network models. In this paper, a convolutional neural network accelerator system composed of RISC-V processor core and Gemmini array accelerator is designed in Chisel language within the Chipyard framework, and the acceleration effect of different Gemmini array configurations for different input matrices is further investigated. The result shows that the accelerator system can achieve thousands of times acceleration compared with a single processor for large matrix calculations.

A 0.67-to-5.4 TSOPs/W Spiking Neural Network Accelerator With 128/256 Reconfigurable Neurons and Asynchronous Fully Connected Synapses

A 0.67-to-5.4 TSOPs/W Spiking Neural Network Accelerator With 128/256 Reconfigurable Neurons and Asynchronous Fully Connected Synapses

Hardware acceleration of Tiny YOLO deep neural networks for sign language recognition: A comprehensive performance analysis

In this paper, we benchmark two automation frameworks, Vitis AI and FINN, for sign language recognition on a Field Programmable Gate Array (FPGA). We conducted an in-depth exploration of both frameworks using Tiny YOLOv2 networks by varying design parameters such as precision, parallelism ratio, etc. Further, a fair baseline comparison is made based on accuracy, speed, and hardware resources. Experimental findings demonstrate that the Vitis AI outperforms the FINN framework and traditional GPU and CPU platforms by achieving significant improvements of 1.08x, 1.7x, and 2.9x in terms of latency. Leveraging Vitis AI, our system achieved a detection speed of 32.7 frames per second (FPS) on the Kria KV260 FPGA with a power consumption rate of 5.6 W and an impressive mean Average Precision (mAP) score of 61.2% on the Hindi Indian Sign Language (ISL) dataset.

Device Specifications for Neural Network Training with Analog Resistive Cross‐Point Arrays Using Tiki‐Taka Algorithms

Recently, specialized training algorithms for analog cross‐point array‐based neural network accelerators have been introduced to counteract device non‐idealities such as update asymmetry and cycle‐to‐cycle variation, achieving software‐level performance in neural network training. However, a quantitative analysis of how these algorithms affect the relaxation of device specifications is yet to be conducted. This study provides a detailed analysis by elucidating the device prerequisites for training with the Tiki‐Taka algorithm versions 1 (TTv1) and 2 (TTv2), which leverage the dynamics between multiple arrays to compensate for device non‐idealities. A multiparameter simulation is conducted to assess the impact of device non‐idealities, including asymmetry, retention, number of pulses, and cycle‐to‐cycle variation, on neural network training. Using pattern‐recognition accuracy as a performance metric, the required device specifications for each algorithm are revealed. The results demonstrate that the standard stochastic gradient descent algorithm requires stringent device specifications. Conversely, TTv2 permits more lenient device specifications than the TTv1 across all examined non‐idealities. The analysis provides guidelines for the development, optimization, and utilization of devices for high‐performance neural network training using Tiki‐Taka algorithms.