Multiply-accumulate Research Articles

Demonstration of Multiply-Accumulate Operation With 28 nm FeFET Crossbar Array

This letter reports a linear multiply-accumulate (MAC) operation conducted on a crossbar memory array based on 28nm high-k metal gate (HKMG) Complementary Metal Oxide Semiconductor (CMOS) and ferroelectric field effect transistor (FeFET). The fabrication is conducted at GlobalFoundries with their standard 28nm technology. The crossbar arrays show a 100% yield in MAC operation on a 300mm wafer. The arrays were divided into <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${8}\times {8}$ </tex-math></inline-formula> segments. The FeFET crossbar arrays were fabricated with access transistors, current-limiter transistors and a current-mode analog-to-digital converter (ADC) on the same wafer. Finally, the data retention characteristics reveal excellent data retention characteristics up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${5}\times {10} ^{{4}}$ </tex-math></inline-formula> seconds, which makes this memory array suitable for carrying out MAC operations in inference engine applications.

Binarized Neural Network Accelerator Macro Using Ultralow-Voltage Retention SRAM for Energy Minimum-Point Operation

A binarized neural network (BNN) accelerator based on a processing-in-memory (PIM)/ computing-in-memory (CIM) architecture using ultralow-voltage retention static random access memory (ULVR-SRAM) is proposed for the energy minimum-point (EMP) operation. The BNN accelerator (BNA) macro is designed to perform stable inference operations at EMP and substantive power-gating (PG) using ULVR at an ultralow voltage (< EMP), which can be applied to fully connected layers (FCLs) with arbitrary shapes and sizes. The EMP operation of the BNA macro, which is enabled by applying the ULVR-SRAM to the macro, can dramatically improve the energy efficiency (TOPS/W) and significantly enlarge the number of parallelized multiply–accumulate (MAC) operations. In addition, the ULVR mode of the BNA macro, which also benefits from the usage of ULVR-SRAM, is effective at reducing the standby power. The proposed BNA macro can show a high energy efficiency of 65 TOPS/W for FCLs. This BNA macro concept using the ULVR-SRAM can be expanded to convolution layers, where the EMP operation is also expected to enhance the energy efficiency of convolution layers.

An Energy-Efficient Mixed-Bitwidth Systolic Accelerator for NAS-Optimized Deep Neural Networks

Optimized deep neural network (DNN) models and energy-efficient hardware designs are of great importance in edge-computing applications. The neural architecture search (NAS) methods are employed for DNN model optimization with mixed-bitwidth networks. To satisfy the computation requirements, mixed-bitwidth convolution accelerators are highly desired for low-power and high-throughput performance. There exist several methods to support mixed-bitwidth multiply-accumulate (MAC) operations in DNN accelerator designs. The low-bitwidth-combination (LBC) method improves the low-bitwidth throughput with a large hardware cost. The high-bitwidth-split (HBS) method minimizes the additional logic gates for configuration. However, the throughput performance in the low-bitwidth mode is poor. In this work, a bit-split-and-combination (BSC) systolic accelerator is proposed. The BSC-based MAC unit is designed to support mixed-bitwidth operations with the best overall performance. Besides, interprocessing element (PE) systolic and intra-PE paralleled dataflow not only improves throughput performance in mixed-bitwidth modes, but also saves power performance for data transmission. The proposed work is designed and synthesized in a 28-nm process. The BSC MAC unit achieves a maximum <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.08\times $ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.75\times $ </tex-math></inline-formula> energy efficiency improvement than the HBS and LBC unit, respectively. Compared with the state-of-the-art accelerators, the proposed work also achieves excellent energy-efficient performance with 20.02, 23.55, and 30.17 TOPS/W on mixed-bitwidth VGG-16, ResNet-18, and LeNet-5 benchmarks at 0.6 V, respectively.

Optimization of Delay IIN Pipeline Mac Unit Using Wallace Tree Multiplier

Abstract: A prompt pipelined MAC unit is implemented in this paper. Due to carrying propagations in additions, including additions of multiplications and additions of accumulations, we will experience a significant path delay as well as a significant increase in power consumption when using the conventional methodology. To resolve these issues, we are including a portion of improvements to the partial product reduction procedure. The pipelined MAC implementation does not complete the addition and accumulation of the most weighted bits until the partial product reduction step of the subsequent multiplication. In order to manage the overspill bits during the partial product reduction procedure, we are designing the small size adder which computes the total number of carries. By comparing with the conventional methodologies, we will get the diminished area as well as power consumption for the proposed pipelined MAC implementation.

Scale-CIM: Precision-scalable computing-in-memory for energy-efficient quantized neural networks

Quantized neural networks (QNNs), which perform multiply-accumulate (MAC) operations with low-precision weights or activations, have been widely exploited to reduce energy consumption. QNNs usually have a trade-off between energy consumption and accuracy depending on the quantized precision, so that it is necessary to select an appropriate precision for energy efficiency. Nevertheless, the conventional hardware accelerators such as Google TPU are typically designed and optimized for a specific precision (e.g., 8-bit), which may degrade energy efficiency for other precisions. Though an analog-based computing-in-memory (CIM) technology supporting variable precision has been proposed to improve energy efficiency, its implementation requires extremely large and power-consuming analog-to-digital converters (ADCs).In this paper, we propose Scale-CIM, a precision-scalable CIM architecture which supports MAC operations based on digital computations (not analog computations). Scale-CIM performs binary MAC operations with high parallelism, by executing digital-based multiplication operations in the CIM array and accumulation operations in the peripheral logic. In addition, Scale-CIM supports multi-bit MAC operations without ADCs, based on the binary MAC operations and shift operations depending on the precision. Since Scale-CIM fully utilizes the CIM array for various quantized precisions (not for a specific precision), it achieves high compute-throughput. Consequently, Scale-CIM enables precision-scalable CIM-based MAC operations with high parallelism. Our simulation results show that Scale-CIM achieves 1.5∼15.8 × speedup and reduces system energy consumption by 53.7∼95.7% across different quantized precisions, compared to the state-of-the-art precision-scalable accelerator.

Optimal Weight‐Splitting in Resistive Random Access Memory‐Based Computing‐in‐Memory Macros

Computing‐in‐memory (CIM) is considered a feasible solution to the acceleration of multiply‐accumulate (MAC) operations at low power. The key to CIM is parallel MAC operations in the memory domain, and thus reductions in power consumption and memory‐access latency. Resistive random access memory (RRAM) can be a good candidate for the memory for CIM given its data nonvolatility, high data density, low‐latency read‐out, multilevel representation, and inherent current accumulation capability. Particularly, the last two attributes offer analog MAC operations in parallel in the memory domain. However, the fully analog MAC operation scheme causes significant power and area overheads for its peripheral circuits, particularly, analog‐to‐digital converters. To compensate for these downsides using digital processing, a method for sub‐array‐wise partial MAC operations over weight‐resistors that are optimally split to minimize power and area overheads for the peripheral circuits is proposed. The simulations performed highlight the optimal sub‐array of in size. That is, weight‐splitting such that a singlew‐bit weight is represented by RRAM cells, i.e., 2‐bit for each cell. For 8‐bit weights, the figure of merit (FOM) for this optimal case reaches ≈28.3 FOM for the case of no weight‐splitting.

HZO-based FerroNEMS MAC for in-memory computing

This paper demonstrates a hafnium zirconium oxide (HZO)-based ferroelectric NEMS unimorph as the fundamental building block for very low-energy capacitive readout in-memory computing. The reported device consists of a 250×30 μm2 unimorph cantilever with 20-nm-thick ferroelectric HZO on 1 μm SiO2. Partial ferroelectric switching in HZO achieves analog programmable control of the piezoelectric coefficient (d31), which serves as the computational weight for multiply accumulate (MAC) operations. The displacement of the piezoelectric unimorph was recorded by actuating the device with different input voltages Vin. The resulting displacement was measured as a function of the ferroelectric programming/poling voltage VP. The slopes of central beam displacement (δmax) vs Vin were measured to be between 182.9 nm/V (for −8 Vp) and −90.5 nm/V (for 8 Vp), which corresponds to displacement proportionality constant β of 68 nm/V2 for +ve Vp and 47 nm/V2 for −ve Vp, demonstrating linear behavior of the multiplier unit. The resultant δmax from AC actuation is in the range of −18 to 36 nm and is a scaled product of Vin and programmed d31 (governed by the Vp). The multiplication function serves as the fundamental unit for MAC operations with the ferroelectric NEMS unimorph. The displacement from many such beams can be added by summing the capacitance changes, providing a pathway to implement a multi-input and multi-weight neuron. A scaling and fabrication analysis suggests that this device can be CMOS compatible, achieving high in-memory computational throughput.

Energy-Efficient DNN Inference on Approximate Accelerators Through Formal Property Exploration

Deep neural networks (DNNs) are being heavily utilized in modern applications, putting energy-constraint devices to the test. To bypass high energy consumption issues, approximate computing has been employed in DNN accelerators to balance out the accuracy-energy reduction trade-off. However, the approximation-induced accuracy loss can be very high and drastically degrade the performance of the DNN. Therefore, there is a need for a fine-grain mechanism that would assign specific DNN operations to approximation to maintain acceptable DNN accuracy, while achieving low energy consumption. We present an automated framework for weight-to-approximation mapping through formal property exploration for approximate DNN accelerators. At the MAC unit level, our experimental evaluation surpassed already energy-efficient mappings by more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 2$ </tex-math></inline-formula> in terms of energy gains, while supporting a fine-grain control over the introduced approximation.

Review and Analysis of Variable Bit-precision MAC Microarchitectures for Energy-efficient AI Computation

In this paper, we review and analyze the state-of-the-art bit-precision scalable multiply-accumulate (MAC) unit microarchitectures. To enable fast and energy-efficient neural network computation, many previous MAC microarchitectures have been explored, but it is hard to select an optimal approach because those works were usually evaluated in different experimental conditions. We first review various variable-bit MAC microarchitectures. Then, we synthesize the variable-bit hardware microarchitectures in a 28 nm technology, and we evaluate the synthesized designs using our custom architecture simulator. Using real benchmarks, we compare various bit-scalable hardware topologies to help researchers choose the most suitable microarchitecture for variable-bit hardware design.

FPGA Implementation of Fast Binary Multiplication Based on Customized Basic Cells

Multiplication is considered one of the most time-consuming and a key operation in wide variety of embedded applications. Speeding up this operation has a significant impact on the overall performance of these applications. A vast number of multiplication approaches are found in the literature where the goal is always to achieve a higher performance. One of these approaches relies on using smaller multiplier blocks which are built based on direct Boolean algebra equations to build large multipliers. In this work, we present a methodology for designing binary multipliers where different sizes customized partial products generation (CPPG) cells are designed and used as smaller building blocks. The sizes of the designed CPPG cells are 2&amp;times;2, 3&amp;times;3, 4&amp;times;4, 5&amp;times;5, and 6&amp;times;6. We use these cells to build 8&amp;times;8, 16&amp;times;16, 32&amp;times;32, 64&amp;times;64, and 128&amp;times;128 binary multipliers. All of the CPPG cells and the binary multipliers are described using the VHDL language, tested, and implemented using XILINX ISE 14.6 tools targeting different FPGA families. The implementation results show that the best performance is achieved when cell 3&amp;times;3 is used and Virtex-7 FPGA is targeted. The binary multipliers that are designed using the proposed CPPG cells achieve better performance when compared with the binary multipliers presented in the literature. As an application that utilizes the proposed multiplier, a Multiply-Accumulate (MAC) unit is designed and implemented in Spartan-3E. The implementation results of the MAC unit demonstrate the effectiveness of the proposed multiplier.

Stateful Rotor for Continuity of Quaternion and Fast Sensor Fusion Algorithm Using 9-Axis Sensors.

Advances in micro-electro-mechanical systems technology have led to the emergence of compact attitude measurement sensor products that integrate acceleration, magnetometer, and gyroscope sensors on a single chip, making them important devices in the field of three-dimensional (3D) attitude measurement for unmanned aerial vehicles, smartphones, and other devices. Sensor fusion algorithms for posture measurement have become an indispensable technology in cutting-edge research, such as human posture measurement using wearable sensors, and stabilization problems in robot position and posture measurement. We have also developed wearable sensors and powered suits in our previous research. We needed a technology for the real-time measurement of a 3D human body motion. It is known that quaternions can be used to algebraically handle 3D rotations; however, sensor fusion algorithms for three sensors are presently complex. This is because these algorithms deal with the post-rotation attitude (pure quaternions) rather than rotation information (the rotor) to avoid a double covering problem involving the rotor. If we are dealing with rotation, it may be possible to make the algorithm simpler and faster by dealing directly with the rotor. In this study, to solve the double covering problem involving the rotor, we propose a stateful rotor and develop a technique for uniquely determining the time-varying states of the rotor. The proposed stateful rotor guarantees the continuity of the rotor parameters with respect to angular changes, and this paper confirms its effectiveness by simulating two rotations around an arbitrary axis. In addition, we verify experimentally that a fast sensor fusion method using stateful rotor can be used for attitude calculation. Experiments also confirm that the calculated results converge to the desired rotation angle for two spatial rotations around an arbitrary axis. Since the proposed stateful rotor extends and stabilizes the definition of the rotor, it is applicable to any algorithm that deals with time-varying quaternionic rotors. In this research, an algorithm based on a multiply–add operation is designed to reduce computational complexity as a high-speed calculation for embedded systems. This method is theoretically equivalent to other methods, while contributing to power saving and the cost reduction of products.

A CNN Hardware Accelerator Using Triangle-based Convolution

Convolutional neural networks (CNNs) have gained a massive impression in the fields of computer vision and especially in the embedded applications because of their high accuracy and performance. However, high computational complexity and power consumption due to convolution operations causes a high demand for low-power accelerators. A 3D geometric optimization strategy is proposed to alleviate the area and power requirements of Multiply Accumulate operations prevalent in all spatial CNNs. The proposed technique is generic and may be easily scaled for accelerators performing spatial 2D convolution.

NoD: A Neural Network-Over-Decoder for Edge Intelligence

The ubiquitous applications of the Internet of Things (IoT) devices and the increasing computational capabilities of neural networks (NNs) have led to a new era of edge computing and a paradigm known as edge intelligence (EI). With EI, the goal is to maximize the utilization of resources available within an edge device, offloading only the most compute-intensive operations to the cloud. In this article, we propose to leverage the close similarity between the internal architecture of a typical network decoder and an NN for deep learning on decoders. The proposed NN-over-decoder is developed in Verilog and synthesized on field-programmable gate array (FPGA). Based on experimental results, the system exhibits power consumption of the same order of magnitude as a baseline decoder and negligible memory overhead while increasing local hardware utilization, alleviating the high communication load in typical communication devices, and offering scalable multiply–accumulate (MAC)/cycle performance compared with the state of the art.

A Codesigned Integrated Photonic Electronic Neuron

In the modern era of artificial intelligence, increasingly sophisticated artificial neural networks (ANNs) are implemented, which pose challenges in terms of execution speed and power consumption. To tackle this problem, recent research on reduced-precision ANNs opened the possibility to exploit analog hardware for neuromorphic acceleration. In this scenario, photonic-electronic engines are emerging as a short-medium term solution to exploit the high speed and inherent parallelism of optics for linear computations needed in ANN, while resorting to electronic circuitry for signal conditioning and memory storage. In this paper we introduce a precision-scalable integrated Photonic-Electronic Multiply-Accumulate Neuron (PEMAN). The proposed device relies on (i) an analog photonic engine to perform reduced-precision multiplications at high speed and low power, and (ii) an electronic front-end for accumulation and application of the nonlinear activation function by means of a nonlinear encoding in the analog-to-digital converter (ADC). The device has been numerically validated through cosimulations to perform multiply-accumulate operations (MAC). Simulations are based on the iSiPP50G SOI process for the photonic engine and a commercial 28 nm CMOS process for the electronic front-end. The PEMAN exhibits a multiplication accuracy of 6.1 ENOB up to 10 GMAC/s, while it can perform computations up to 56 GMAC/s with a reduced accuracy down to 2.1 ENOB. The device can trade off speed and power consumption with resolution, significantly outperforming its analog electronics counterparts both in terms of speed and energy consumption. With respect to other photonic ANNs, the PEMAN has comparable speed and energy consumption with a higher resolution, while outperforming them by a hundredfold in the fan-in, which opens the possibility to accelerate more complex networks.

Design of Power and Area Efficient Approximate Multipliers for Edge Detection Algorithm

Abstract: In this paper, we present a novel approximate computing scheme suitable for realizing the energy-efficient multiplyaccumulate (MAC) processing. First we design the approximate 4-2 compressors generating errors in the opposite direction while minimizing the computational costs. Based on the probabilistic analysis, positive and negative multipliers are then carefully developed to provide a similar error distance. Simulation results on various practical applications reveal that the proposed MAC processing offers the energy-efficient computing scenario by extending the range of approximate parts. This Design is implemented by Verilog HDL and simulated by Modelsim 6.4 c. The Performance is measured by Xilinx tool Synthesis Process. The proposed Sobel edge detection algorithm uses approximation methods to replace the complex operations; This design is done by Matlab and Modelsim using hdldameon, This proposed multipliers arerpelaced in the sobel operator based image Edge detection.

SwAFL: A Library of High-Performance Activation Function for the Sunway Architecture

The Sunway supercomputers have recently attracted considerable attention to execute neural networks. Meanwhile, activation functions help extend the applicability of neural networks to nonlinear models by introducing nonlinear factors. Despite the numerous activation function-supported AI frameworks, only PyTorch and TensorFlow were ported to the Sunway platforms. Although these libraries can meet the minimum functional requirements to deploy a neural network on the Sunway machines, there still exist some drawbacks including the limited number of usable functions and unsatisfactory performances remaining unresolved. Therefore, two activation function algorithms with different computing accuracies were developed in this study, and an efficient implementation scheme was designed using the single instruction/multiple data extension and multiply–add instructions of the platform. Finally, an efficient library-swAFL-composed of 48 function interfaces was designed and implemented on the Sunway platforms. Experimental results indicate that swAFL outperformed PyTorch and TensorFlow by 19.5 and 23 times, respectively, on average.

P2OP—Plant Pathology on Palms: A deep learning-based mobile solution for in-field plant disease detection

Plant diseases are one of the dominant factors that threaten sustainable agriculture, leading to economic losses. Developing an accurate mobile-based plant disease detection methodology is important for enabling rapid identification of emerging diseases directly from the farms. The deep learning methods have limited usage in mobile-based applications as they require larger memory and processing power to operate directly on smartphones or internet connectivity when used with a client–server computing model. To address this challenge, we propose a mobile-based lightweight deep learning-based model, which requires only a small footprint and processing power while maintaining higher detection accuracy. With around 0.088 billion multiply–accumulation operations, 0.26 million parameters, and 1 MB storage space, this framework achieved 97%, 97.1% and 96.4% accuracies on apple, citrus and tomato leaves datasets, respectively. One of our tiny models achieved 93.33% accuracy on a custom sourced in-the-wild apple leaves images dataset, which affirms the in-field applicability of the proposed framework. The superiority of the proposed model is further demonstrated through a comparative study with equivalent lightweight models.

Energy-efficient neural network design using memristive MAC unit

Artificial intelligence applications implemented with neural networks require extensive arithmetic capabilities through multiply-accumulate (MAC) units. Traditional designs based on voltage-mode circuits feature complex logic chains for such purposes as carry processing. Additionally, as a separate memory block is used (e.g., in a von Neumann architecture), data movements incur on-chip communication bottlenecks. Furthermore, conventional multipliers have both operands encoded in the same physical quantity, which is either low cost to update or low cost to hold, but not both. This may be significant for low-energy edge operations. In this paper, we propose and present a mixed-signal multiply-accumulate unit design with in-memory computing to improve both latency and energy. This design is based on a single-bit multiplication cell consisting of a number of memristors and a single transistor switch (1TxM), arranged in a crossbar structure implementing the long-multiplication algorithm. The key innovation is that one of the operands is encoded in easy to update voltage and the other is encoded in non-volatile memristor conductance. This targets operations such as machine learning which feature asymmetric requirements for operand updates. Ohm’s Law and KCL take care of the multiplication in analog. When implemented as part of a NN, the MAC unit incorporates a current to digital stage to produce multi-bit voltage-mode output, in the same format as the input. The computation latency consists of memory writing and result encoding operations, with the Ohm’s Law and KCL operations contributing negligible delay. When compared with other memristor-based multipliers, the proposed work shows an order of magnitude of latency improvement in 4-bit implementations partly because of the Ohm’s Law and KCL time savings and partly because of the short writing operations for the frequently updated operand represented by voltages. In addition, the energy consumption per multiplication cycle of the proposed work is shown to improve by 74%–99% in corner cases. To investigate the usefulness of this MAC design in machine learning applications, its input/output relationships is characterized using multi-layer perceptrons to classify the well-known hand-writing digit dataset MNIST. This case study implements a quantization-aware training and includes the non-ideal effect of our MAC unit to allow the NN to learn and preserve its high accuracy. The simulation results show the NN using the proposed MAC unit yields an accuracy of 93%, which is only 1% lower than its baseline.

A neural network method for time-dependent inverse source problem with limited-aperture data

This paper is concerned with the mathematical design of a novel gesture-based input/instruction device with identification and prediction using a moving emitter. The emitter acts as a point source and there is a sensor monitoring the wave field emitted by the emitter away from it on a surface in real time. In practice, the emitter could be a ring worn on a finger of the human being who desires to interact/communicate with the computer, while the sensor could be mounted on a computer. The input/instruction can be recognized and predicted by identifying the moving trajectory of the emitter performed by the human being from the collected wave field data. The process can be modeled as an inverse moving source problem, that is, one identifies and predicts the trajectory of a moving point source by measuring the corresponding wave field. There are several salient features of our study. First, for the practical consideration, the dynamical wave field data are collected in a limited aperture and full aperture respectively. Second, we design a parameter inversion model by neural network (PIMNN) to reconstruct the trajectory of the moving point source. This model solves the problem of information loss caused by data acquisition in limited aperture and has certain robustness with respect to noise. The computing complexity of the PIMNN are calculated by the Multiply Accumulate. Third, we consider the trajectory prediction of the moving point source for the inverse source problem associated with the novel input/instruction approach, and construct a trajectory prediction model by neural network (TPMNN) to predict the trajectory of the moving point source. Numerical experiments show that the proposed device works effectively and efficiently in some practical scenarios.

An Aqueous Analog MAC Machine.

Using ions in aqueous milieu for signal processing, like in biological circuits, may potentially lead to a bioinspired information processing platform. Studies, however, have focused on individual ionic diodes and transistors rather than circuits comprising many such devices. Here a 16×16 array of new ionic transistors is developed in an aqueous quinone solution. Each transistor features a concentric ring electrode pair with a disk electrode at the center. The electrochemistry of these electrodes in the solution provides the basis for the transistor operation. The ring pair electrochemically tunes the local electrolytic concentration to modulate the disk's Faradaic reaction rate. Thus, the disk current as a Faradaic reaction to the disk voltage is gated by the ring pair. The 16×16 array of these transistors performs analog multiply-accumulate (MAC) operations, a computing modality hotly pursued for low-power artificial neural networks. This exploits the transistor's operating regime where the disk current is a multiplication of the disk voltage and a weight parameter tuned by the ring pair gating. Such disk currents from multiple transistors are summated in a global reference electrode to complete a MAC task. This ionic circuit demonstrating analog computing is a step toward sophisticated aqueous ionics.