Convolutional Neural Network Processor Research Articles

A Novel Instruction Driven 1-D CNN Processor for ECG Classification.

Electrocardiography (ECG) has emerged as a ubiquitous diagnostic tool for the identification and characterization of diverse cardiovascular pathologies. Wearable health monitoring devices, equipped with on-device biomedical artificial intelligence (AI) processors, have revolutionized the acquisition, analysis, and interpretation of ECG data. However, these systems necessitate AI processors that exhibit flexible configuration, facilitate portability, and demonstrate optimal performance in terms of power consumption and latency for the realization of various functionalities. To address these challenges, this study proposes an instruction-driven convolutional neural network (CNN) processor. This processor incorporates three key features: (1) An instruction-driven CNN processor to support versatile ECG-based application. (2) A Processing element (PE) array design that simultaneously considers parallelism and data reuse. (3) An activation unit based on the CORDIC algorithm, supporting both Tanh and Sigmoid computations. The design has been implemented using 110 nm CMOS process technology, occupying a die area of 1.35 mm2 with 12.94 µW power consumption. It has been demonstrated with two typical ECG AI applications, including two-class (i.e., normal/abnormal) classification and five-class classification. The proposed 1-D CNN algorithm performs with a 97.95% accuracy for the two-class classification and 97.9% for the five-class classification, respectively.

Assessment of Winograd-based processor element for CNN processor and ASIC implementation of convolution processor element

Assessment of Winograd-based processor element for CNN processor and ASIC implementation of convolution processor element

A 0.57 mW@1 FPS In-Column Analog CNN Processor Integrated Into CMOS Image Sensor

A 0.57 mW@1 FPS In-Column Analog CNN Processor Integrated Into CMOS Image Sensor

Assessment of Winograd-based processor element for CNN processor and ASIC implementation of convolution processor element

The convolutional neural network (CNN) is the most widely used machine learning technique within the fields of image and video processing. It is primarily used to categorize images using vast datasets. This require a lot of calculations. The effectiveness of a field-programmable gate array (FPGA) as a hardware accelerator for CNNs will give excellent performance at low power budgets. The employment of the Winograd algorithm can reduce the number of processing stages in CNN. 2-D convolution is employed for the bulk of calculations in CNNs. The tactic for computing convolution for smaller filter sizes that uses Winograd minimum filtering is the handiest. The comparison of computation complexity for multiplications will be performed using Matlab. The architecture of the Winograd-based processing element, RTL coding in Verilog HDL, and the test bench are designed to examine the performance. The Xilinx Vivado / Cadence tool will be used to implement the processing element in the convolution unit on an FPGA or ASIC.

Design of Convolutional Neural Network Processor Based on FPGA Resource Multiplexing Architecture.

As CNNs are widely used in fields such as image classification and target detection, the total number of parameters and computation of the models is gradually increasing. In addition, the requirements on hardware resources and power consumption for deploying CNNs are becoming higher and higher, leading to CNN models being restricted to certain specific platforms for miniaturization and practicality. Therefore, this paper proposes a convolutional-neural-network-processor design with an FPGA-based resource-multiplexing architecture, aiming to reduce the consumption of hardware resources and power consumption of CNNs. First, this paper takes a handwritten-digit-recognition CNN as an example of a CNN design based on a resource-multiplexing architecture, and the prediction accuracy of the CNN can reach 97.3 percent by training and testing with Mnist dataset. Then, the CNN is deployed on FPGA using the hardware description language Verilog, and the design is optimized by resource multiplexing and parallel processing. Finally, the total power consumption of the system is 1.03 W and the power consumption of the CNN module is 0.03 W under the premise of guaranteeing the prediction accuracy, and the prediction of a picture is about 68,139 clock cycles, which is 340.7 us under a 200 MHz clock. The experimental results have obvious advantages in terms of resources and power consumption compared with those reported in related articles in recent years, and the design proposed in this paper.

A reconfigurable processor for mix-precision CNNs on FPGA

To solve the problem of low computing efficiency of existing accelerators for convolutional neural network (CNNs), which caused by the inability to adapt to the characteristics of computing mode and caching for the mixed-precision quantized CNNs model, we propose a reconfigurable CNN processor in this paper, which consists of the reconfigurable adaptable computing unit, flexible on-chip cache unit and macro-instruction set. The multi-core CNN processor can be reconstructed according to the structure of CNN models and constraints of reconfigurable resources, to improve the utilization of computing resources. The elastic on-chip buffer and the data access approach by dynamically configuring an address to better utilization of on-chip memory. Then, the macroinstruction set architecture (mISA) can fully express the characteristics of the mixed-precision CNN models and reconfigurable processors, to reduce the complexity of mapping CNNs with different network structures and computing modes to reconfigurable the CNNs processors. For the well-known CNNs-VGG16 and ResNet-50, the proposed CNN processor has been implemented using Ultra96-V2 and ZCU102 FPGA, showing the throughput of 216.6 GOPS, and 214 GOPS, the computing efficiency of 0.63 GOPS/DSP and 0.64 GOPS/DSP on Ultra96-V2, respectively, achieving a better efficiency than the CNN accelerator based on fixed bit-width. Meanwhile, for ResNet-50, the throughput and the computing efficiency are up to 931.8 GOPS, 0.40 GOPS/DSP on ZCU102, respectively. In addition, these achieve up to 55.4% higher throughput than state-of-the-art CNN accelerators.

WITHDRAWN: A 30 MB re-configurable convolutional neural network processor for high-performance and energy-efficient operation

Technology scaling has driven computing devices to be faster, cheaper, and smaller while consuming less power in past decades. However, as technology scaling has be- come increasingly difficult in recent years, power has become the major constraint in performance, and thus, the improvement in the performance of mobile devices has begun to diminish. Moreover, emerging intelligent mobile systems are demanding increasing computing power. In light of this challenge associated with artificial intelligence (AI), domain-specific architectures are widely believed to be the path to realizing considerable improvements in the efficiency, performance and cost of intelligent mobile systems. This work presents several algorithm, architecture and circuit optimized solutions for intelligent and autonomous mobile systems, including vision-based stereo depth, optical flow, simultaneous localization and mapping (SLAM) and convolutional neural network (CNN) based image recognition. The prototypes are implemented for demonstration and verification. It is an ReRAM-CNN processor that enables semantic understanding through machine learning. The work exploits various optimizations including parallelism, scheduling, exploiting scarcity and circuit customization to overcome the complexity of these problems for extremely energy-efficient, real-time, robust operation. The impact is significant in the age of AI as mobile systems can become increasingly intelligent in daily life, powered by these proposed solutions.

DT-CNN: An Energy-Efficient Dilated and Transposed Convolutional Neural Network Processor for Region of Interest Based Image Segmentation

An energy-efficient convolutional neural network (CNN) processor is proposed for real-time image segmentation on mobile devices. The proposed processor utilizes Region of Interest (ROI) based image segmentation to speed up the process and reduce the overall external memory access. Although the ROI based image segmentation degrades the segmentation accuracy, the proposed dilation rate adjustment algorithm, which regulates the receptive field depending on the ROI resolution during dilated convolution, compensates for the accuracy degradation up to 0.2310 mean Intersection over Union (mIoU). In addition, the processor accelerates the dilated and transposed convolution by skipping the redundant zero computations with the proposed delay cells. As a result, the throughput of dilated and transposed convolution is increased up to $\times 159$ and $\times 3.84$ . The delay cells can also support the variable dilation rates in dilated convolution caused by the dilation rate adjustment algorithm. Moreover, the processor selects the operating frequency based on the ROI resolution to save power consumption up to 81.2%. The processor is simulated in 65 nm CMOS technology, and the 6.8 mm2 processor consumes the 206 mW power consumption with the 4.66 ms of processing time and 3.22 TOPS/W energy-efficiency at the target image segmentation dataset.

Rank order coding based spiking convolutional neural network architecture with energy-efficient membrane voltage updates

Spiking neural network (SNN) system that uses rank order coding (ROC) as input spike encoding, generally suffers from low recognition accuracy and unnecessary computations that increase complexities. In this paper, we present a Spiking convolutional neural network (Spiking CNN) architecture that significantly improves recognition accuracy as well as computation efficiencies based on a novel ROC and modified kernel sizes. The proposed ROC generates spike trains based on maximum input value without sorting operations. In addition, as the recognition accuracy is affected by the reduced number of spikes as layers become deeper, the proposed ROC is inserted just before the final layer to increase the number of input spikes. The 2 × 2 pooling kernels are also replaced with 4 × 4 to reduce the network size. The hardware architecture of the proposed Spiking CNN has been implemented using 65 nm CMOS process. Neuron-centric membrane voltage update approach is also efficiently exploited in convolutional and fully connected layers to improve the hardware energy efficiencies. The Spiking CNN processor is seamlessly processing 2.85 K classifications per second with 6.79 uJ/classification. It also achieves 90.2% of recognition accuracy for MNIST dataset using unsupervised learning with STDP.

A Power-Efficient CNN Accelerator With Similar Feature Skipping for Face Recognition in Mobile Devices

A low power face recognition (FR) convolutional neural network (CNN) processor is proposed with high power efficiency to achieve always-on FR in mobile devices. Three key features enable a power-efficient FR CNN. First, tile-based clustering (THC) is proposed for reducing the computation overhead of hierarchical clustering. It generates an average of 37.2% duplicated input features in the entire network. Second, a low latency tile-based hierarchical clustering core is proposed. It supports an approximated clustering method that removes distance updates and increases pipeline utilization by up to 98.6%. Finally, a similar feature skipping binary convolution core with separated accumulation cores is proposed to increase power efficiency through skipping redundant MAC operation between duplicated input features and weights. It can skip 38.9% of CNN operations and reduce 35.8% of the total computation considering both CNN and THC. The proposed features enable the proposed processor to operate FR CNN with 17.3 TOPS/W power efficiency, which is $1.3\times $ higher compared to the previous state-of-the-art FR CNN processor. Implemented in 65nm CMOS technology, the 6mm2 FR CNN processor shows 0.5mW power consumption at 1 frames-per-second always-on face recognition.

MulNet: A Flexible CNN Processor With Higher Resource Utilization Efficiency for Constrained Devices

Leveraging deep convolutional neural networks (DCNNs) for various application areas has become a recent inclination of many machine learning practitioners due to their impressive performance. Research trends show that the state-of-the-art networks are getting deeper and deeper and such networks have shown significant performance increase. Deeper and larger neural networks imply the increase in computational intensity and memory footprint. This is particularly a problem for inference-based applications on resource constrained computing platforms. On the other hand, field-programmable gate arrays (FPGAs) are becoming a promising choice in giving hardware solutions for most deep learning implementations due to their high-performance and low-power features. With the rapid formation of various state-of-the-art CNN architectures, a flexible CNN hardware processor that can handle different CNN architectures and yet customize itself to achieve higher resource efficiency and optimum performance is critically important. In this paper, a novel and highly flexible DCNN processor, MulNet, is proposed. MulNet can be used to process most regular state-of-the-art CNN variants aiming at maximizing resource utilization of a target device. A processing core with multiplier and without multiplier is employed to achieve that. We formulated optimum fixed-point quantization format for MulNet by analyzing layer-by-layer quantization error. We also created a power-of-2 quantization for multiplier-free (MF) processing core of MulNet. Both quantizations significantly reduced the memory space needed and the logic consumption in the target device. We utilized Xilinx Zynq SoCs to leverage the one die hybrid (CPU and FPGA) architecture. We devised a scheme that utilizes Zynq processing system (PS) for memory intensive layers and the Zynq programmable logic (PL) for computationally intensive layers. We implemented modified LeNet, CIFAR-10 full, ConvNet processor (CNP), MPCNN, and AlexNet to evaluate MulNet. Our architecture with MF processing cores shows the promising result, by saving 36%-72% on-chip memory and 10%-44% DSP48 IPs, compared to the architecture with cores implemented using the multiplier. Comparison with the state of the art showed a very promising 25-$40\times $ DSP48 and 25-$29\times $ on-chip memory reduction with up to 136.9-GOP/s performance and 88.49-GOP/s/W power efficiency. Hence, our results demonstrate that the proposed architecture can be very expedient for resource constrained devices.

A Low-Power Convolutional Neural Network Face Recognition Processor and a CIS Integrated With Always-on Face Detector

A Low-power convolutional neural network (CNN)-based face recognition system is proposed for the user authentication in smart devices. The system consists of two chips: an always-on CMOS image sensor (CIS)-based face detector (FD) and a low-power CNN processor. For always-on FD, analog–digital Hybrid Haar-like FD is proposed to improve the energy efficiency of FD by 39%. For low-power CNN processing, the CNN processor with 1024 MAC units and 8192-bit-wide local distributed memory operates at near threshold voltage, 0.46 V with 5-MHz clock frequency. In addition, the separable filter approximation is adopted for the workload reduction of CNN, and transpose-read SRAM using 7T SRAM cell is proposed to reduce the activity factor of the data read operation. Implemented in 65-nm CMOS technology, the $3.30 \times 3.36$ mm2 CIS chip and the $4 \times 4$ mm2 CNN processor consume 0.62 mW to evaluate one face at 1 fps and achieved 97% accuracy in LFW dataset.

Low-Power Convolutional Neural Network Processor for a Face-Recognition System

The authors propose a low-power convolutional neural network (CNN)-based face recognition system for user authentication in smart devices. The system comprises two chips: an always-on functional CMOS image sensor (CIS) for imaging and face detection (FD) and a low-power CNN processor (CNNP) for face verification (FV). A functional CIS integrated with an FD accelerator enables event-driven chip-to-chip communication for face images only when there is a face. To achieve low power consumption in FD while maintaining the memory size required for the FD processing not to exceed the on-chip memory size, the authors present two-stage FD using an analog FD unit and a digital FD unit. For the event-driven FV, the CNNP adopts dynamic voltage and frequency scaling to minimize the power consumption when the number of faces in input images changes dynamically. In addition, tensor decomposition is used to reduce a CNN's workload, and the CNNP architecture based on transpose-read SRAM (T-SRAM) allows low power consumption by reducing the local memory access. Implemented in 65-nm CMOS technology, the 3.30 3.36 mmsup2/sup functional CIS and the 4 4 mmsup2/sup CNNP consume 0.62 mW to evaluate one face at 1 frame per second and achieve 97 percent accuracy in the LFW dataset.

The training of convolution neural network for advanced driver assistant system

In this paper, the learning technique for CNN processor on vehicle is proposed. In the case of conventional CNN processors, weighted values learned through training are stored for use, but when there is distortion in the image due to the weather conditions, the accuracy is decreased. Therefore, the method of enhancing the input image for classification is general, but it has the weakness of increasing the processor size. To solve this problem, the CNN performance was improved in this paper through the learning method of the distorted image. As a result, the proposed method showed improvement of approximately 38% better accuracy than the conventional method.

A 2.71 nJ/Pixel Gaze-Activated Object Recognition System for Low-Power Mobile Smart Glasses

A low-power object recognition (OR) system with intuitive gaze user interface (UI) is proposed for battery-powered smart glasses. For low-power gaze UI, we propose a low-power single-chip gaze estimation sensor, called gaze image sensor (GIS). In GIS, a novel column-parallel pupil edge detection circuit (PEDC) with new pupil edge detection algorithm XY pupil detection (XY-PD) is proposed which results in 2.9× power reduction with 16× larger resolution compared to previous work. Also, a logarithmic SIMD processor is proposed for robust pupil center estimation, (1 pixel error, with low-power floating-point implementation. For OR, low-power multicore OR processor (ORP) is implemented. In ORP, task-level pipeline with keypoint-level scoring is proposed to reduce the number of cores as well as the operating frequency of keypoint-matching processor (KMP) for low-power consumption. Also, dual-mode convolutional neural network processor (CNNP) is designed for fast tile selection without external memory accesses. In addition, a pipelined descriptor generation processor (DGP) with LUT-based nonlinear operation is newly proposed for low-power OR. Lastly, dynamic voltage and frequency scaling (DVFS) for dynamic power reduction in ORP is applied. Combining both of the GIS and ORP fabricated in 65 nm CMOS logic process, only 75 mW average power consumption is achieved with real-time OR performance, which is 1.2× and 4.4× lower power than the previously published work.

Reaction-Diffusion Navigation Robot Control: From Chemical to VLSI Analogic Processors

We introduce a new methodology and experimental implementations for real-time wave-based robot navigation in a complex, dynamically changing environment. The main idea behind the approach is to consider the robot arena as an excitable medium, in which moving objects-obstacles and the target-are represented by sites of autowave generation: the target generates attractive waves, while the obstacles repulsive ones. The moving robot detects traveling and colliding wave fronts and uses the information about dynamics of the autowaves to adapt its direction of collision-free motion toward the target. This approach allows us to achieve a highly adaptive robot behavior and thus an optimal path along which the robot reaches the target while avoiding obstacles. At the computational and experimental levels, we adopt principles of computation in reaction-diffusion (RD) nonlinear active media. Nonlinear media where autowaves are used for information processing purposes can therefore be considered as RD computing devices. In this paper, we design and experiment with three types of RD processors: experimental and computational Belousov-Zhabotinsky chemical processor, computational CNN processor, and experimental RD-CNN very large-scale integration chip-the complex analog and logic computing engine (CACE1k). We demonstrate how to experimentally implement robot navigation using space-time snapshots of active chemical medium and how to overcome low-speed limitation of this wetware implementation in CNN-based silicon processors.