Multiply-accumulate Research Articles

CSGN: Combined Channel- and Spatial-Wise Dynamic Gating Architecture for Convolutional Neural Networks

The explosive computation and memory requirements of convolutional neural networks (CNNs) hinder their deployment in resource-constrained devices. Because conventional CNNs perform identical parallelized computations even on redundant pixels, the saliency of various features in an image should be reflected for higher energy efficiency and market penetration. This paper proposes a novel channel and spatial gating network (CSGN) for adaptively selecting vital channels and generating spatial-wise execution masks. A CSGN can be characterized as a dynamic channel and a spatial-aware gating module by maximally utilizing opportunistic sparsity. Extensive experiments were conducted on the CIFAR-10 and ImageNet datasets based on ResNet. The results revealed that, with the proposed architecture, the amount of multiply-accumulate (MAC) operations was reduced by 1.97–11.78× and 1.37–13.12× on CIFAR-10 and ImageNet, respectively, with negligible accuracy degradation in the inference stage compared with the baseline architectures.

A reconfigurable high-speed and low-complexity residue number system-based multiply-accumulate channel filter for software radio receivers

PurposeFilter design plays an essential role in most communication standards. The essential element of the software-defined radio is a channelizer that comprises several channel filters. Designing filters with lower complexity, minimized area and enhanced speed is a demanding task in currently prevailing communication standards. This study aims to propose an efficient reconfigurable residue number system (RNS)-based multiply-accumulate (MAC) channel filter for software radio receivers.Design/methodology/approachRNS-based pipelined MAC module for the realization of channel finite impulse response (FIR) filter architecture is considered in this work. Further, the use of a single adder and single multiplier for realizing the filter architecture regardless of the number of taps offers effective resource sharing. This design provides significant improvement in speed of operation as well as a reduction in area complexity.FindingsIn this paper, two major tasks have been considered: first, the RNS number conversion is performed in which the integer is converted into several residues. These residues are processed in parallel and are applied to the MAC-FIR filter architecture. Second, the MAC filter architecture involves pipelining, which enhances the speed of operation to a significant extent. Also, the time-sharing-based design incorporates a single partial product-based shift and add multiplier and single adder, which provide a low complex design. The results show that the proposed 16-tap RNS-based pipelined MAC sub-filter achieves significant improvement in speed as well as 89.87% area optimization when examined with the conventional RNS-based FIR filter structure.Originality/valueThe proposed MAC-FIR filter architecture provides good performance in terms of complexity and speed of operation because of the use of the RNS scheme with pipelining and partial product-based shift and adds multiplier and single adder when examining with the conventional designs. The reported architecture can be used in software radios.

Quantization and sparsity-aware processing for energy-efficient NVM-based convolutional neural networks

Nonvolatile memory (NVM)-based convolutional neural networks (NvCNNs) have received widespread attention as a promising solution for hardware edge intelligence. However, there still exist many challenges in the resource-constrained conditions, such as the limitations of the hardware precision and cost and, especially, the large overhead of the analog-to-digital converters (ADCs). In this study, we systematically analyze the performance of NvCNNs and the hardware restrictions with quantization in both weight and activation and propose the corresponding requirements of NVM devices and peripheral circuits for multiply–accumulate (MAC) units. In addition, we put forward an in situ sparsity-aware processing method that exploits the sparsity of the network and the device array characteristics to further improve the energy efficiency of quantized NvCNNs. Our results suggest that the 4-bit-weight and 3-bit-activation (W4A3) design demonstrates the optimal compromise between the network performance and hardware overhead, achieving 98.82% accuracy for the Modified National Institute of Standards and Technology database (MNIST) classification task. Moreover, higher-precision designs will claim more restrictive requirements for hardware nonidealities including the variations of NVM devices and the nonlinearities of the converters. Moreover, the sparsity-aware processing method can obtain 79%/53% ADC energy reduction and 2.98×/1.15× energy efficiency improvement based on the W8A8/W4A3 quantization design with an array size of 128 × 128.

The deep multichannel discrete‐time cellular neural network model for classification

High latency and power consumption are two major problems that need to be addressed in convolutional neural networks (CNN). In this paper, the convolutional layer is replaced with a discrete-time cellular neural network (CellNN) to overcome these problems. Multiple configurations of CellNNs are trained in a framework called TensorFlow to classify objects from the CIFAR-10 database. Effects of the number of iterations, the number of channels, batch normalization, and activation functions on the classification accuracies are presented. It is shown that TensorFlow is a tool that is capable of training discrete-time CellNNs. Although the accuracies of the proposed networks on CIFAR-10 are slightly lesser than the existing CNNs, with reduced parameters and multiply-accumulates (MACs), power consumption and computation time of our networks will be less than CNNs.

Hardware Realization of Low Power and Area Efficient Vedic Mac in DSP Filters

Abstract: VLSI experiences a key position in many of the signal process applications. Multiply and Accumulation methodology is onein all told the chiefly used operation. Power, space and speed ar the metrics accustomed ensure the efficiency of a MAC unit. Surely cases each of these metrics plays a key role. In some cases, speed is simply targeted, so the other parameters do not appear to be rich priority in that case. Through the deep analysis of adders, Carry Select Adder has shown less space and power consumption than totally different adders. The processes that square measure involved in MAC are multiplication, addition and accumulation. The addition of Vedic techniques in a MAC is commonly an additional advantage .So, this project includes development of multiply and accumulate unit pattern frightened writing Sanskrit (UrdhvaTiryakbhyam sutra), accumulation unit involving Carry select adder (CSLA) and its implementation in a 4-tap FIR filter

Optical processor for a binarized neural network.

We propose and experimentally demonstrate an optical processor for a binarized neural network (NN). Implementation of a binarized NN involves multiply-accumulate operations, in which positive and negative weights should be implemented. In the proposed processor, the positive and negative weights are realized by switching the operations of a dual-drive Mach-Zehnder modulator (DD-MZM) between two quadrature points corresponding to two binary weights of +1 and -1, and the multiplication is also performed at the DD-MZM. The accumulation operation is realized by dispersion-induced time delays and detection at a photodetector (PD). A proof-of-concept experiment is performed. A binarized convolutional neural network (CNN) accelerated by the optical processor at a speed of 32 giga floating point operations/s (GFLOPS) is tested on two benchmark image classification tasks. The large bandwidth and parallel processing capability of the processor has high potential for next generation data computing.

Fast and Accurate Approximation Methods for Trigonometric and Arctangent Calculations for Low-Performance Computers

In modern computers, complicated signal processing is highly optimized with the use of compilers and high-speed processing using floating-point units (FPUs); therefore, programmers have little opportunity to care about each process. However, a highly accurate approximation can be processed in a small number of computation cycles, which may be useful when embedded in a field-programmable gate array (FPGA) or micro controller unit (MCU), or when performing many large-scale operations on a graphics processing unit (GPU). It is necessary to devise algorithms to obtain the desired calculated values without an accelerator or compiler assistance. The residual correction method (RCM) developed here can produce simple and accurate approximations of certain nonlinear functions with minimal multiply–add operations. In this study, we designed an algorithm for the approximate computation of trigonometric and inverse trigonometric functions, which are nonlinear elementary functions, to achieve their fast and accurate computation. A fast first approximation and a more accurate second approximation of each function were created using RCM with a less than 0.001 error using multiply–add operations only. This achievement is particularly useful for MCUs, which have a low power consumption but limited computational power, and the proposed approximations are candidate algorithms that can be used to stabilize the attitude control of robots and drones, which require real-time processing.

Prior wavelet knowledge for multi-modal medical image segmentation using a lightweight neural network with attention guided features

Medical image segmentation plays a crucial role in diagnosing and staging diseases. It facilitates image analysis and quantification in multiple applications, but building the right appropriate solutions is essential and highly reliant on the features of different datasets and computational resources. Most existing approaches provide segmentation for a specific anatomical region of interest and are limited to multiple imaging modalities in a clinical setting due to their generalizability with high computational requirements. To mitigate these issues, we propose a robust and lightweight deep learning real-time segmentation network for multi-modality medical images called MISegNet. We incorporate discrete wavelet transform (DWT) of the input to extract salient features in the frequency domain. This mechanism allows the neurons’ receptive field to enlarge within the network. We propose a self-attention-based global context-aware (SGCA) module with varying dilation rates to enlarge the field of view and designate the importance of each scale that enhances the network’s ability to discriminate features. We build a residual shuffle attention (RSA) mechanism to improve the feature representation of the proposed model and formulate a new boundary-aware loss function called Farid End Point Error (FEPE) that correctly segments regions with ambiguous boundaries for edge detection. We confirm the versatility of the proposed model by performing experiments against eleven state-of-the-art segmentation methods on four datasets of different organs, including two publicly available datasets (i.e., ISBI2017, and COVID-19 CT) and two private datasets (i.e., ovary and liver ultrasound images). Experimental results prove that the MISegNet with 1.5M parameters, outperforms the state-of-the-art methods by 1.5%–7% (i.e., dice coefficient score) with a corresponding 23× decrease in the number of parameters and multiply-accumulate operations respectively compared to U-Net.

Channel-Wise Average Pooling and 1D Pixel-Shuffle Denoising Autoencoder for Electrode Motion Artifact Removal in ECG

This paper presents a channel-wise average pooling and one dimension pixel-shuffle architecture for a denoising autoencoder (CPDAE) design that can be applied to efficiently remove electrode motion (EM) artifacts in an electrocardiogram (ECG) signal. The three advantages of the proposed design are as follows: (1) In the skip connection layer, less memory is needed to transfer the features extracted by the neural network; (2) Pixel shuffle and pixel unshuffle techniques with point-wise convolution are used to effectively reserve the key features generated from each layer in both the encoder and decoder; (3) Overall, fewer parameters are required to reconstruct the ECG signal. This paper describes three deep neural network models, namely CPDAELite, CPDAERegular, and CPDAEFull, which support various computational capacity and hardware arrangements. The three proposed structures involve an encoder and decoder with six, seven, and eight layers, respectively. Furthermore, the CPDAELite, CPDAERegular, and CPDAEFull structures require fewer multiply-accumulate operations—355.01, 56.96, and 14.69 million, respectively—and less parameter usage—2.69 million, 149.7 thousand, and 55.5 thousand, respectively. To evaluate the denoising performance, the MIT–BIH noise stress test database containing six signal-to-noise ratios (SNRs) of noisy ECGs was employed. The results demonstrated that the proposed models had a higher improvement of SNR and lower percentage root-mean-square difference than other state-of-the-art methods under various conditions of SNR.

Stacked Reconfigurable Optical Cavities for Smart Sensing Pixels

Photonic systems have seen a recent explosion of investigation for neuromorphic engineering and the execution of machine learning models, owing to the advantages of optics for communications and performing linear operations. Here, we present a novel architecture for executing reconfigurable multiwavelength photonic multiply-accumulate (MAC) operations based on all-solid-state vertically-integrated tunable cavities. We simulate this weighting action through full transfer-matrix calculations of a realistic candidate filter stack. By efficiently leveraging the out-of-plane dimension, we form a conceptual ‘smart pixel’ device that can achieve higher spectral channel count per area, or equivalently computational density, than comparable subpixel architectures and planar integrated photonic devices. While this technology is of use for neuromorphic photonic information processing, its usefulness extends to applications in sensing and communications by covering, for instance, high spatial resolution hyperspectral filters and reconfigurable routing.

Power-Efficient Deep Neural Network Accelerator Minimizing Global Buffer Access without Data Transfer between Neighboring Multiplier—Accumulator Units

This paper presents a novel method for minimizing the power consumption of weight data movements required by a convolutional operation performed on a two-dimensional multiplier–accumulator (MAC) array of a deep neural-network accelerator. The proposed technique employs a local register file (LRF) at each MAC unit in a manner such that once weight pixels are read from the global buffer into the LRF, they are reused from the LRF as many times as desired instead of being repeatedly fetched from the global buffer in each convolutional operation. One of the most evident merits of the proposed method is that the procedure is completely free from the burden of data transfer between neighboring MAC units. It was found from our simulations that the proposed method provides a power saving of approximately 83.33% and 97.62% compared with the power savings recorded by the conventional methods, respectively, when the dimensions of the input data matrix and weight matrix are 128 × 128 and 5 × 5, respectively. The power savings increase as the dimensions of the input data matrix or weight matrix increase.

Low-overhead inverted LUT design for bounded DNN activation functions on floating-point vector ALUs

An inference engine uses floating-point numbers to provide high accuracy in deep neural network computing despite its computing resource limitations. However, the computation for non-linear activation functions occurs the performance bottleneck, and we may alleviate it by adopting a lookup table (LUT) method. However, the floating-point number system’s characteristic, where intervals between mantissa numbers differ depending on their exponent values, makes it challenging to calculate LUT index values and produce the error-tolerant outputs.This paper proposes a floating-point-based lookup table (FP-LUT) that produces minimal errors and requires negligible hardware cost, especially for vector arithmetic logic units (ALUs), using bfloat16 recently proposed for both inference and training processes. Instead of calculating the index using the function input value, we apply the principle of an inverse function for our design, especially targeting bounded DNN activation functions. We divide a range of function output values linearly by the number of LUT entry sizes and store the corresponding input values in the LUT. Then, we compare the incoming input value with the stored LUT values, find the corresponding address, and convert it into an FP format for the output. We applied our 32-entry FP-LUT to the in-house 8-way bfloat16 MAC unit to support four DNN activation functions: logistic sigmoid, hyper-tangent, soft sign, and ISRU, which incurs only 1.22% and 0.46% of the area and power consumption overhead. Our accuracy analysis shows that with only an entry size of 1/8 compared to state-of-the-art 16-bit fixed-point LUT methods and the small logic overhead, FP-LUT reduces the average errors by 51.8%, 28.4%, 14.4%, and 26.1% in those functions on our test datasets, respectively. Additionally, we show that our scheme satisfies all application-defined accuracy.

Silicon-Based Metastructure Optical Scattering Multiply-Accumulate Computation Chip.

Optical neural networks (ONN) have become the most promising solution to replacing electronic neural networks, which have the advantages of large bandwidth, low energy consumption, strong parallel processing ability, and super high speed. Silicon-based micro-nano integrated photonic platforms have demonstrated good compatibility with complementary metal oxide semiconductor (CMOS) processing. Therefore, without completely changing the existing silicon-based fabrication technology, optoelectronic hybrid devices or all-optical devices of better performance can be achieved on such platforms. To meet the requirements of smaller size and higher integration for silicon photonic computing, the topology of a four-channel coarse wavelength division multiplexer (CWDM) and an optical scattering unit (OSU) are inversely designed and optimized by Lumerical software. Due to the random optical power splitting ratio and incoherency, the intensities of different input signals from CWDM can be weighted and summed directly by the subsequent OSU to accomplish arbitrary multiply–accumulate (MAC) operations, therefore supplying the core foundation for scattering ONN architecture.

Towards Convolutional Neural Network Acceleration and Compression Based on Simonk-Means.

Convolutional Neural Networks (CNNs) are popular models that are widely used in image classification, target recognition, and other fields. Model compression is a common step in transplanting neural networks into embedded devices, and it is often used in the retraining stage. However, it requires a high expenditure of time by retraining weight data to atone for the loss of precision. Unlike in prior designs, we propose a novel model compression approach based on Simon k-means, which is specifically designed to support a hardware acceleration scheme. First, we propose an extension algorithm named Simon k-means based on simple k-means. We use Simon k-means to cluster trained weights in convolutional layers and fully connected layers. Second, we reduce the consumption of hardware resources in data movement and storage by using a data storage and index approach. Finally, we provide the hardware implementation of the compressed CNN accelerator. Our evaluations on several classifications show that our design can achieve 5.27× compression and reduce 74.3% of the multiply–accumulate (MAC) operations in AlexNet on the FASHION-MNIST dataset.

A 0.02% Accuracy Loss Voltage-Mode Parallel Sensing Scheme for RRAM-Based XNOR-Net Application

Compute-in-memory (CiM) is an effective way to solve the memory wall issue. As a promising candidate for CiM, resistive random access memory (RRAM) has the advantages of non-volatile, fast operating speed, and programmable resistance. However, the resistance variation and poor retention will cause computation accuracy loss, especially in neural networks’ multiply-accumulate (MAC). In this brief, we propose a voltage-mode XNOR logic mapping scheme to implement MAC which could realize the linear change of the voltage on the source line (SL) of the RRAM array. The scheme was verified on a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$32\times 32$ </tex-math></inline-formula> RRAM array with a binary neural network (BNN) for MNIST and CIFAR-10 classification. The implementation results show only 0.14% accuracy loss (MINST) compared with software implementation and an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8.7\times $ </tex-math></inline-formula> improvement in energy consumption compared with current-mode sensing. In addition, we further propose an energy-efficient parallel scheme to reduce the accuracy loss to 0.02% (MINST) and achieve a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4.34\times $ </tex-math></inline-formula> improvement in energy consumption.

A Weighted Current Summation Based Mixed Signal DRAM-PIM Architecture for Deep Neural Network Inference

Processing-in-Memory (PIM) is an emerging approach to bridge the memory-computation gap. One of the major challenges of PIM architectures in the scope of Deep Neural Network (DNN) inference is the implementation of area-intensive Multiply-Accumulate (MAC) units in memory technologies, especially for DRAM-based PIMs. The DRAM architecture restricts the integration of DNN computation units near the area optimized commodity DRAM Sub-Array (SA) or Primary Sense Amplifier (PSA) region, where the data parallelism is maximum and the data movement cost is minimum. In this paper, we present a novel DRAM-based PIM architecture that is based on bit-decomposed MAC operation and Weighted Current Summation (WCS) technique to implement the MAC unit with minimal additional circuitry in the PSA region by leveraging on mixed-signal design. The architecture presents a two-stage design that employs light-weight current mirror based analog units near the SAs in the PSA region, whereas all the other substantial logic is integrated near the bank peripheral region. Hence, our architecture attains a balance between the data parallelism, data movement energy and area optimization. For an 8-bit CNN inference, our novel 8Gb DRAM PIM device achieves a peak performance of 142.8GOPS while consuming a power of 756.76mW, which results in an energy efficiency of 188.8GOPS/W. The area overhead of such an 8Gb device for a 2ynm DRAM technology is 12.63% in comparison to a commodity 8Gb DRAM device.

Posit Process Element for Using in Energy-Efficient DNN Accelerators

In this work, we present an energy-efficient posit processing element (PE) for utilization in array-based deep neural network (DNN) accelerators along with an approximation method for further reducing the energy consumption of the unit. The posit arithmetic used in the proposed PE provides high precision for the considered data widths even when approximation is used for operations. Using some modification/simplification approaches and proposing a speculative posit adder (SPA) unit, we reduce the complexity of the employed posit multiply–accumulator (MAC) in the proposed PE. The effectiveness of the proposed PE is studied using a 45-nm CMOS technology. The results reveal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3.5\times $ </tex-math></inline-formula> and 92% improvements in the delay and energy consumption, respectively, compared to those of the state-of-the-art posit PE. To assess the efficacy of the proposed PE, we have modeled an 8-bit DNN accelerator and employed it for the implementation of some DNN architectures. The results indicate that the proposed PE and its approximate one provide, on average, 19.3% and 29.6% lower energy consumptions compared to that of the latest prior work when providing 10.6% and 5.8% higher accuracies, respectively.

D-NAT: Data-Driven Non-Ideality Aware Training Framework for Fabricated Computing-In-Memory Macros

To enable energy-efficient computation for deep neural networks (DNNs) at edge, computing-in-memory (CIM) is proposed to reduce the energy costs during intense off-chip memory access. However, CIM is prone to multiply-accumulate (MAC) errors due to non-idealities of memory crossbars and peripheral circuits, which severely degrade the accuracy of DNNs. In this work, we propose a Data-Driven Non-ideality Aware Training (D-NAT) framework to compensate for the accuracy degradation. The proposed D-NAT framework has the following contributions: 1) We measured a fabricated SRAM-based CIM macro to obtain a data-driven MAC error model (D-MAC-EM). Based on the derived D-MAC-EM, we analyze the impact of the non-idealities on DNN’s accuracy. 2) To make DNNs robust to the non-idealities of CIM macros, we incorporate the measured D-MAC-EM into DNN’s training procedure. Moreover, we propose a statistical training mechanism to better estimate the gradients of the discrete D-MAC-EM. 3) We investigate trade-offs between quantization range and quantization errors. To mitigate the quantization errors in activations, we introduce extended PACT (E-PACT) that adaptively learns the upper and lower bounds of input activations for each layer. Simulation results show that our proposed D-NAT improves the accuracy of ResNet20, VGG8, ResNet34, and VGG16 by 78.98%, 71.8%, 72.04%, and 57.85%, respectively, which reaches the ideal upper bound of the quantized model. Lastly, the D-NAT framework is validated on an FPGA platform with the fabricated SRAM-based CIM macro chip. Based on the measurement results, D-NAT successfully recovers the accuracy under non-idealities of a real SRAM-based CIM macro.

Design and implementation of 32-bit MAC unit using reversible logic gate

Design and implementation of 32-bit MAC unit using reversible logic gate

Resource-constrained FPGA implementation of YOLOv2

Progress is being made to deploy convolutional neural networks (CNNs) into the Internet of Things (IoT) edge devices for handling image analysis tasks locally. These tasks require low-latency and low-power computation on low-resource IoT edge devices. However, CNN-based algorithms, e.g. YOLOv2, typically contain millions of parameters. With the increase in the CNN’s depth, filters are increased by a power of two. A large number of filters and operations could lead to frequent off-chip memory access that affects the operation speed and power consumption of the device. Therefore, it is a challenge to map a deep CNN into a low-resource edge IoT platform. To address this challenge, we present a resource-constrained Field-Programmable Gate Array implementation of YOLOv2 with optimized data transfer and computing efficiency. Firstly, a scalable cross-layer dataflow strategy is proposed which allows on-chip data transfer between different types of layers, and offers flexible off-chip data transfer when the intermediate results are unaffordable on-chip. Next, a filter-level data-reuse dataflow strategy together with a filter-level parallel multiply-accumulate operation computing processing elements array is developed. Finally, multi-level sliding buffers are developed to optimize the convolutional computing loop and reuse the input feature maps and weights. Experiment results show that our implementation has achieved 4.8 W of low-power consumption for executing YOLOv2, an 8-bit deep CNN containing 50.6 MB weights, using low-resource of 8.3 Mbits on-chip memory. The throughput and power efficiency are 100.33 GOP/s and 20.90 GOP/s/W, respectively.