Neural Network Accelerator Research Articles

Fast-OverlaPIM: A Fast Overlap-Driven Mapping Framework for Processing In-Memory Neural Network Acceleration

Fast-OverlaPIM: A Fast Overlap-Driven Mapping Framework for Processing In-Memory Neural Network Acceleration

Advanced Preliminary Screening for PD Detection for Overhead Distribution Lines with Covered Conductors

This study proposes a novel approach to optimize partial discharge detection in overhead lines with covered conductors. The approach uses 2D Convolutional Neural Networks and HW neural network accelerator Edge TPU. The critical advancement of this research lies in exploring two distinct 2D classification methods - 2D histograms and spectrogram - and adjusting neural network thresholds to selectively identify potential positive PD samples with enhanced accuracy. The proposed approach addresses the prevalent challenge of costly and limited remote data transmission in PD detection. It significantly reduces the need for extensive data transfer by focusing on potentially positive samples. This evaluation of two approaches for detection of partial discharges, coupled with the strategic threshold adjustment, presents a novel solution in the realm of PD detection, offering increased efficiency and cost-effectiveness. By evaluating and comparing these strategies of 2D classification and threshold optimization, this research contributes to the field of partial discharges detection. It proposes a method that not only minimizes operational costs but also aligns with environmental sustainability goals, paving the way for more advanced maintenance practices in power transmission systems.

SC-PLR: An Approximate Spiking Neural Network Accelerator with On-Chip Predictive Learning Rule

SC-PLR: An Approximate Spiking Neural Network Accelerator with On-Chip Predictive Learning Rule

EncodingNet: A Novel Encoding-based MAC Design for Efficient Neural Network Acceleration

EncodingNet: A Novel Encoding-based MAC Design for Efficient Neural Network Acceleration

An Efficient FPGA-Based Dilated and Transposed Convolutional Neural Network Accelerator

An Efficient FPGA-Based Dilated and Transposed Convolutional Neural Network Accelerator

Deep neural networks accelerators with focus on tensor processors

The massive amount of data and the problem of processing them is one of the main challenges of the digital age, and the development of artificial intelligence and machine learning can be useful in solving this problem. Using deep neural networks to improve the efficiency of these two areas is a good solution. So far, several architectures have been introduced for data processing with the benefit of deep neural networks, whose accuracy, efficiency, and computing power are different from each other. This article tries to review these architectures, their features, and their functions in a systematic way. According to the current research style, 24 articles (conference and research articles related to this topic) have been evaluated in the period of 2014–2022. In fact, the significant aspects of the selected articles are compared and at the end, the upcoming challenges and topics for future research are presented. The results show that the main parameters for proposing a new tensor processor include increasing speed and accuracy and reducing data processing time, reducing on-chip storage space, reducing DRAM access, reducing energy consumption, and achieving high efficiency.

Mapping of Deep Neural Network Accelerators on Wireless Multistage Interconnection NoCs

In the last few decades, the concept of Wireless Network-on-chip (WiNoC) has emerged as a promising alternative for Multiprocessor Systems on Chip (MPSOC) to achieve reliable and scalable communication. Worth recalling in this regard is that our research team has already designed, verified and evaluated Multistage Interconnection Networks (MIN) in this field. With respect to the present work, we consider proceeding with further exploring our thoughts on this research area. Firstly, we propose the design and performance evaluation of a hybrid (wireless/wired) MIN, analysing how this augmented network can potentially improve not only the average delay, but also energy consumption. Secondly, we continue with examining the implementation of our advanced DELTA-based MIN architecture on Deep Neural Network (DNN) accelerators, while accounting for its potential regularity and scalability in simultaneously maintaining an effective power efficiency and lower latency throughout the DNN operating process. In this context, several metrics have been evaluated in regard to three DNN application cases through implementation of their main respective modules.

Hardware Accelerated Optimization of Deep Learning Model on Artificial Intelligence Chip

With the rapid development of deep learning technology, the demand for computing resources is increasing, and the accelerated optimization of hardware on artificial intelligence (AI) chip has become one of the key ways to solve this challenge. This paper aims to explore the hardware acceleration optimization strategy of deep learning model on AI chip to improve the training and inference performance of the model. In this paper, the method and practice of optimizing deep learning model on AI chip are deeply analyzed by comprehensively considering the hardware characteristics such as parallel processing ability, energy-efficient computing, neural network accelerator, flexibility and programmability, high integration and heterogeneous computing structure. By designing and implementing an efficient convolution accelerator, the computational efficiency of the model is improved. The introduction of energy-efficient computing effectively reduces energy consumption, which provides feasibility for the practical application of mobile devices and embedded systems. At the same time, the optimization design of neural network accelerator becomes the core of hardware acceleration, and deep learning calculation such as convolution and matrix operation are accelerated through special hardware structure, which provides strong support for the real-time performance of the model. By analyzing the actual application cases of hardware accelerated optimization in different application scenarios, this paper highlights the key role of hardware accelerated optimization in improving the performance of deep learning model. Hardware accelerated optimization not only improves the computing efficiency, but also provides efficient and intelligent computing support for AI applications in different fields.

Integrated WDM-compatible optical mode division multiplexing neural network accelerator

On-chip photonic neural networks (PNN) are emerging as an attractive solution for artificial neural networks due to their high computing density, low energy consumption, and compact size. Matrix-vector multiplication (MVM) plays a key role in on-chip PNN, and can achieve high-speed multiply-accumulate operation. Most current schemes implement MVM by adopting wavelength division multiplexing technology to accumulate the power of different wavelengths together. This requires multiple laser sources. Additionally, both positive and negative domain MVM are inevitable for realizing precise PNNs, but because of the innate limitations of light, effective solutions to perform negative value computing are still inadequate. Here, we propose and demonstrate a PNN accelerator based on mode division multiplexing technology to reduce the use of multi-wavelength lasers. We show that it can satisfactorily tackle real-number-field computing (including positive and negative domains) based on a novel, to our knowledge, transformation mapping approach. As a proof-of-concept, we demonstrate a fabricated accelerator for image convolution and letter pattern detection, achieving a computing density of 1.37TOPS/mm2 under the 22.38 Gbaud modulation rate.

WRA-MF: A Bit-Level Convolutional-Weight-Decomposition Approach to Improve Parallel Computing Efficiency for Winograd-Based CNN Acceleration

FPGA-based convolutional neural network (CNN) accelerators have been extensively studied recently. To exploit the parallelism of multiplier–accumulator computation in convolution, most FPGA-based CNN accelerators heavily depend on the number of on-chip DSP blocks in the FPGA. Consequently, the performance of the accelerators is restricted by the limitation of the DSPs, leading to an imbalance in the utilization of other FPGA resources. This work proposes a multiplication-free convolutional acceleration scheme (named WRA-MF) to relax the pressure on the required DSP resources. Firstly, the proposed WRA-MF employs the Winograd algorithm to reduce the computational density, and it then performs bit-level convolutional weight decomposition to eliminate the multiplication operations. Furthermore, by extracting common factors, the complexity of the addition operations is reduced. Experimental results on the Xilinx XCVU9P platform show that the WRA-MF can achieve 7559 GOP/s throughput at a 509 MHz clock frequency for VGG16. Compared with state-of-the-art works, the WRA-MF achieves up to a 3.47×–27.55× area efficiency improvement. The results indicate that the proposed architecture achieves a high area efficiency while ameliorating the imbalance in the resource utilization.

Monolithic 3D stacking for neural network acceleration

Monolithic 3D stacking for neural network acceleration

Empirical evaluation of filter pruning methods for acceleration of convolutional neural network

Empirical evaluation of filter pruning methods for acceleration of convolutional neural network

CoGNN: An Algorithm-Hardware Co-Design Approach to Accelerate GNN Inference With Minibatch Sampling

As a new algorithm of graph embedding, Graph Neural Networks (GNNs) have been widely used in many fields. However, GNN computing has the characteristics of both sparse graph processing and dense neural network, which make it difficult to be deployed efficiently on the existing graph processing accelerators or neural network accelerators. Recently, some GNN accelerators have been proposed, but the following challenges have not been fully solved: 1) The mini-batch GNN inference scenario has the potential of software and hardware co-design, which can bring 30% computation amount reduction, and this is not well utilized. Besides, the cost of Message Flow Graph construction is large and may account for more than 50% of the total delay. 2) The feature aggregation has a large amount of data access and relatively small amount of computation, which leads to low on-chip data reuse, only 10% of dense computing. 3) Without the optimization of sparse computing units, simple memory bank and cross bar architecture can easily lead to bank access conflict and load imbalance, reducing the utilization of computing units to less than 60%. In order to solve the above problems, we propose a algorithm-hardware co-design scheme to accelerate GNN inference, which includes three technologies: 1) A reuse-aware sampling method is proposed for mini-batch inference scenarios, which reduces 30% of the calculation and improves the on-chip reusability of local data. 2) Through the node-wise parallelism-aware quantization, the features and weights are quantized to integers with eight or four bits, which reduces the amount of memory access by at least four times. 3) An accelerator supporting the above technologies is designed and evaluated, and different operations are supported by the sampling-inference integration architecture. The multi-bank on-chip memory pool is designed to support data reuse, and edge stream reordering is used to reduce data access conflicts, improving the utilization of computing units by 1.5x. Combined with the above technologies, the experiments show that our design achieves 9.2× speedup and 29× energy efficiency improvement compared with Deep Graph Library framework running on servers equiped with CPU and GPU.

Acceleration of convolutional neural network based diabetic retinopathy diagnosis system on field programmable gate array

Diabetic retinopathy (DR) is one of the most common causes of blindness. The necessity for a robust and automated DR screening system for regular examination has long been recognized in order to identify DR at an early stage. In this paper, an embedded DR diagnosis system based on convolutional neural networks (CNNs) has been proposed to assess the proper stage of DR. We coupled the power of CNN with transfer learning to design our model based on state-of-the-art architecture. We preprocessed the input data, which is color fundus photography, to reduce undesirable noise in the image. After training many models on the dataset, we chose the adopted ResNet50 because it produced the best results, with a 92.90% accuracy. Extensive experiments and comparisons with other research work show that the proposed method is effective. Furthermore, the CNN model has been implemented on an embedded target to be a part of a medical instrument diagnostic system. We have accelerated our model inference on a field programmable gate array (FPGA) using Xilinx tools. Results have confirmed that a customized FPGA system on chip (SoC) with hardware accelerators is a promising target for our DR detection model with high performance and low power consumption.

CNN Accelerator at the Edge With Adaptive Zero Skipping and Sparsity-Driven Data Flow

An energy-efficient convolutional neural network (CNN) accelerator is proposed for low-power inference on edge devices. An adaptive zero skipping technique is proposed to dynamically skip the zeros in either activations or weights, depending on which has the higher sparsity. The characteristic of non-zero data aggregation is explored to enhance the effectiveness of adaptive zero skipping in performance boosting. To mitigate the load imbalance issue after zero skipping, a sparsity-driven data flow and low-complexity dynamic task allocation are employed for different convolution layers. Facilitated further by a two-stage distiller, the proposed accelerator achieves 5.42×, 3.41×, and 3.42× performance boosting for VGG16, AlexNet, and Mobilenet-v1, respectively, compared to the baseline. Implemented in a 55-nm low power CMOS technology, the proposed accelerator achieves an effective energy efficiency of 2.41 TOPS/W, 2.35 TOPS/W, and 0.64 TOPS/W for VGG16, AlexNet, and Mobilenet-v1, respectively, at 100 MHz and 1.08 V supply voltage.

TREAD-M3D: Temperature-Aware DNN Accelerators for Monolithic 3-D Mobile Systems

Monolithic 3D (MONO3D) integration provides performance and power efficiency benefits over 2D circuits and, thus, is a potent technology for the design of Deep Neural Network (DNN) accelerators with enhanced energy efficiency. However, high IC temperatures are major challenges for the design of MONO3D systems. To this end, this paper focuses on designing temperature-aware MONO3D DNN accelerators. We propose a new automated method, called TREAD-M3D, that provides a near-optimal MONO3D DNN accelerator architecture in terms of systolic array size, SRAM organization, partition across 3D layers, and operating frequency, for a given DNN, optimization goal, and temperature constraint. TREAD-M3D incorporates circuit-and architecture-level models to evaluate the power and performance characteristics of different partitions. Our method reveals valuable insights and enables tradeoff analysis for achieving high energy efficiency in MONO3D systolic arrays. In comparison to recent works that adopt a fixed partition choice to design MONO3D DNN systems, TREAD-M3D yields up to 22% higher energy efficiency. Using TREAD-M3D, we further demonstrate that temperature unawareness not only leads to infeasible configurations due to temperature violations but also over-estimates energy-delay-product benefits by up to 24%.

Partial Sum Quantization for Reducing ADC Size in ReRAM-Based Neural Network Accelerators

Partial Sum Quantization for Reducing ADC Size in ReRAM-Based Neural Network Accelerators

A Simple Nonlinear Classifier Using a Multimode Optical Chip

Neural network accelerator based on photonic‐integrated circuits is emerging as a promising technology for fast, power‐efficient, and parallel computing. Under such technology, optics is mainly used for linear transformations, e.g., through an array of cascaded switching networks. Nonlinear activation is implemented either electronically requiring extra optical–electrical conversion or via nonlinear optical materials that often suffer from high loss, large power consumption, and difficulty in integration. Herein, an optical neural chip with only one multimode waveguide, fabricated using low‐cost linear optical materials, plus seven heater electrodes to control the multimode interference, is proposed. The nonlinear networks are intrinsically integrated in the electrical‐to‐optical signal conversion through the waveguide. The linear computation, in the electronic domain, is included in the mandatory step to convert the input matrix to the intermediate current values on the seven electrodes. Though extremely simple, the proposed system can classify nonlinear datasets and images by optical readout with high accuracy and without calibration. Prospects for future development are given at the end. In this work, an alternative route is offered to exploiting the classic multimode interference for advanced optical computing applications.

NPU-Accelerated Imitation Learning for Thermal Optimization of QoS-Constrained Heterogeneous Multi-Cores

Thermal optimization of a heterogeneous clustered multi-core processor under user-defined QoS targets requires application migration and DVFS. However, selecting the core to execute each application and the VF levels of each cluster is a complex problem because (1) the diverse characteristics and QoS targets of applications require different optimizations, and (2) per-cluster DVFS requires a global optimization considering all running applications. State-of-the-art resource management for power or temperature minimization either relies on measurements that are commonly not available (such as power) or fails to consider all the dimensions of the optimization (e.g., by using simplified analytical models). To solve this, ML methods can be employed. In particular, IL leverages the optimality of an oracle policy, yet at low run-time overhead, by training a model from oracle demonstrations. We are the first to employ IL for temperature minimization under QoS targets. We tackle the complexity by training NN at design time and accelerate the run-time NN inference using NPU. While such NN accelerators are becoming increasingly widespread, they are so far only used to accelerate user applications. In contrast, we use for the first time an existing accelerator on a real platform to accelerate NN-based resource management. To show the superiority of IL compared to RL in our targeted problem, we also develop multi-agent RL-based management. Our evaluation on a HiKey 970 board with an Arm big.LITTLE CPU and NPU shows that IL achieves significant temperature reductions at a negligible run-time overhead. We compare TOP-IL against several techniques. Compared to ondemand Linux governor, TOP-IL reduces the average temperature by up to 17 ˆC at minimal QoS violations for both techniques. Compared to the RL policy, our TOP-IL achieves 63 % to 89 % fewer QoS violations while resulting similar average temperatures. Moreover, TOP-IL outperforms the RL policy in terms of stability. We additionally show that our IL-based technique also generalizes to different software (unseen applications) and even hardware (different cooling) than used for training.

AD 2 VNCS: A dversarial D efense and D evice V ariation-tolerance in Memristive Crossbar-based N euromorphic C omputing S ystems

In recent years, memristive crossbar-based neuromorphic computing systems (NCS) have obtained extremely high performance in neural network acceleration. However, adversarial attacks and conductance variations of memristors bring reliability challenges to NCS design. First, adversarial attacks can fool the neural network and pose a serious threat to security critical applications. However, device variations lead to degradation of the network accuracy. In this article, we propose DFS (Deep neural network Feature importance Sampling) and BFS (Bayesian neural network Feature importance Sampling) training strategies, which consist of Bayesian Neural Network (BNN) prior setting, clustering-based loss function, and feature importance sampling techniques, to simultaneously combat device variation, white-box attack, and black-box attack challenges. Experimental results clearly demonstrate that the proposed training framework can improve the NCS reliability.