Multiply-accumulate Research Articles

Redundancy-free integrated optical convolver for optical neural networks based on arrayed waveguide grating

Abstract Optical neural networks (ONNs) have gained significant attention due to their potential for high-speed and energy-efficient computation in artificial intelligence. The implementation of optical convolutions plays a vital role in ONNs, as they are fundamental operations within neural network architectures. However, state-of-the-art convolution architectures often suffer from redundant inputs, leading to substantial resource waste. Here, we demonstrate an integrated optical convolution architecture that leverages the inherent routing principles of arrayed waveguide grating (AWG) to execute the sliding of convolution kernel and summation of results. M × N multiply–accumulate (MAC) operations are facilitated by M + N units within a single clock cycle, thus eliminating the redundancy. In the experiment, we achieved 5 bit precision and 91.9 % accuracy in the handwritten digit recognition task confirming the reliability of our approach. Its redundancy-free architecture, low power consumption, high compute density (8.53 teraOP mm−1 s−1) and scalability make it a valuable contribution to the field of optical neural networks, thereby paving the way for future advancements in high-performance computing and artificial intelligence applications.

Optimized Multiplier Architectures for Enhanced Performance and Efficiency in MAC Units

This paper investigated the performance of Vedic multipliers in a 32-bit Multiplier-Accumulator Unit (MAC) by comparing Urdhva Tiryakbhyam and Nikhilam Sutras with various adder architectures. The goal was to identify the optimal combination of speed and resource efficiency. Urdhva Tiryakbhyam with CLA emerged as the fastest option, achieving a minimal delay of 0.709 ns. However, this came at the cost of higher resource utilization, measured in Logic Look-Up Tables (LUTs). Conversely, Nikhilam implementations generally required fewer LUTs, making them more resource-efficient, but they exhibited slightly slower performance. CLA consistently delivered the best delay for both Vedic multiplier types among the adder architectures. All the explored configurations are viable for practical implementation on Xilinx ISE 14.7. The key takeaway is that the choice between Urdhva Tiryakbhyam and Nikhilam and the specific adder architecture hinges on the application’s priorities.

An Efficient Human Activity Recognition In-Memory Computing Architecture Development for Healthcare Monitoring.

Human activity recognition has played a crucial role in healthcare information systems due to the fast adoption of artificial intelligence (AI) and the internet of thing (IoT). Most of the existing methods are still limited by computational energy, transmission latency, and computing speed. To address these challenges, we develop an efficient human activity recognition in-memory computing architecture for healthcare monitoring. Specifically, a mechanism-oriented model of Ag/a-Carbon/Ag memristor is designed, serving as the core circuit component of the proposed in-memory computing system. Then, one-transistor-two-memristor (1T2M) crossbar array is proposed to perform high-efficiency multiply-accumulate (MAC) operation and high-density memory in the proposed scheme. To facilitate understanding of the proposed efficient human activity recognition in-memory computing design, self-attention ConvLSTM module, multi-head convolutional attention module, and recognition module are proposed. Furthermore, the proposed system is applied to perform human activity recognition, which contains eleven different human activities, including five different postural falls, and six basic daily activities. The experimental results show that the proposed system has advantages in recognition performance (≥ 0.20% accuracy, ≥ 1.10% F1-score) and time consumption (approximately 8∼10 times speed up) compared to existing methods, indicating an advancement in smart healthcare applications.

A Near-Sensor Processing Accelerator for Approximate Local Binary Pattern Networks

In this work, a high-speed and energy-efficient comparator-based <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u> ear- <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</u> ensor <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</u> ocal <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B</u> inary <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</u> attern accelerator architecture (NS-LBP) is proposed to execute a novel local binary pattern deep neural network. First, inspired by recent LBP networks, we design an approximate, hardware-oriented, and multiply-accumulate (MAC)-free network named Ap-LBP for efficient feature extraction, further reducing the computation complexity. Then, we develop NS-LBP as a processing-in-SRAM unit and a parallel in-memory LBP algorithm to process images near the sensor in a cache, remarkably reducing the power consumption of data transmission to an off-chip processor. Our circuit-to-application co-simulation results on MNIST and SVHN datasets demonstrate minor accuracy degradation compared to baseline CNN and LBP-network models, while NS-LBP achieves 1.25 GHz and an energy-efficiency of 37.4 TOPS/W. NS-LBP reduces energy consumption by 2.2× and execution time by a factor of 4× compared to the best recent LBP-based networks.

A runtime-reconfigurable convolutional engine using tensor multiplication with multiple computing modes in 22-nm CMOS

The ultra-low power consumption and flexible configurability of hardware are urgent for resource-constrained artificial intelligence of things (AIoT). Thus, we propose a convolutional engine using tensor multiplication. It consists of a reconfigurable processing element (RPE) array to dynamically adjust convolutional operations with varying kernel sizes during runtime. Implemented in a 22-nm CMOS process, the proposed RPE cluster achieves high energy efficiency, flexibility, and resource utilization with low-cost hardware overhead as compared to state-of-the-art (SOTA) PE architectures. Furthermore, multiply-accumulate (MAC) operations in RPE support accurate and multiple approximate computing modes. Approximate modes achieve a configurable approximation degree, coupled with a search strategy for the approximation factor to improve energy efficiency. In the neural network-based keyword spotting (KWS) task, RPE cluster with the proposed approximation solution saves the whole system's power consumption by 34.64 % with only 0.7 % accuracy loss as compared to using accurate computing modes altogether. It achieves the lowest inference energy as compared to the SOTA.

Integrated WDM-compatible optical mode division multiplexing neural network accelerator

On-chip photonic neural networks (PNN) are emerging as an attractive solution for artificial neural networks due to their high computing density, low energy consumption, and compact size. Matrix-vector multiplication (MVM) plays a key role in on-chip PNN, and can achieve high-speed multiply-accumulate operation. Most current schemes implement MVM by adopting wavelength division multiplexing technology to accumulate the power of different wavelengths together. This requires multiple laser sources. Additionally, both positive and negative domain MVM are inevitable for realizing precise PNNs, but because of the innate limitations of light, effective solutions to perform negative value computing are still inadequate. Here, we propose and demonstrate a PNN accelerator based on mode division multiplexing technology to reduce the use of multi-wavelength lasers. We show that it can satisfactorily tackle real-number-field computing (including positive and negative domains) based on a novel, to our knowledge, transformation mapping approach. As a proof-of-concept, we demonstrate a fabricated accelerator for image convolution and letter pattern detection, achieving a computing density of 1.37TOPS/mm2 under the 22.38 Gbaud modulation rate.

WRA-MF: A Bit-Level Convolutional-Weight-Decomposition Approach to Improve Parallel Computing Efficiency for Winograd-Based CNN Acceleration

FPGA-based convolutional neural network (CNN) accelerators have been extensively studied recently. To exploit the parallelism of multiplier–accumulator computation in convolution, most FPGA-based CNN accelerators heavily depend on the number of on-chip DSP blocks in the FPGA. Consequently, the performance of the accelerators is restricted by the limitation of the DSPs, leading to an imbalance in the utilization of other FPGA resources. This work proposes a multiplication-free convolutional acceleration scheme (named WRA-MF) to relax the pressure on the required DSP resources. Firstly, the proposed WRA-MF employs the Winograd algorithm to reduce the computational density, and it then performs bit-level convolutional weight decomposition to eliminate the multiplication operations. Furthermore, by extracting common factors, the complexity of the addition operations is reduced. Experimental results on the Xilinx XCVU9P platform show that the WRA-MF can achieve 7559 GOP/s throughput at a 509 MHz clock frequency for VGG16. Compared with state-of-the-art works, the WRA-MF achieves up to a 3.47×–27.55× area efficiency improvement. The results indicate that the proposed architecture achieves a high area efficiency while ameliorating the imbalance in the resource utilization.

Fault Diagnosis for Modular Multilevel Converter (MMC) Based on Deep Learning: An Edge Implementation Using Binary Neural Network

Recently deep neural network (DNN) has been proposed for fault diagnosis of modular multilevel converter (MMC). The training of DNN is convenient due to its end-to-end feature, and DNN has a strong learning ability due to its deep structure. However, DNN is hard to be implemented at the edge (i.e., the MMC side) since it involves a large number of floating-point parameters and calculations. This paper proposes a methodology to deploy a DNN with fault diagnosis purpose at the edge. First, the floating-point DNN is converted to a binary neural network (BNN) version. BNN not only binarizes the floating-point weights as &#x201C;1&#x201D; or &#x201C;&#x2013;1&#x201D; to save the storage footprint, but also replaces the floating-point multiply-accumulates (MACs) with bitwise operations to reduce the computation complexity. Second, the computation of different BNN layers is distributed to different embedded real-time controllers in MMC, so as to fully use the existing computing resources. With the proposed methodology, a typical DNN, namely one-dimensional convolutional neural network (1-D CNN) for open-circuit fault (OCF) diagnosis of MMC, which has 238.93kB parameter storage footprint and 91,752 floating-point MACs, is replaced by a BNN with only 7.48kB parameter storage footprint and bitwise operations. 96.87% storage footprint is saved and almost floating-point operations are replaced, and as a tradeoff, the diagnosis accuracy is only reduced by 4.33% compared with the 1-D CNN counterpart. This work shows the huge potential of edge intelligence for fault diagnosis of power electronics.

A Brain-Inspired Hierarchical Interactive In-Memory Computing System and Its Application in Video Sentiment Analysis

Video sentiment analysis can effectively establish the relationship between the emotion state and the multimodal information, while still suffer from intensive computation and low efficiency, due to the von Neumann computing architecture. Here, we present a brain-inspired hierarchical interactive in-memory computing (IMC) system, which can efficiently solve ‘von Neumann bottleneck’, enabling cross-modal interactions and semantic gap elimination. First, a 1T1M synapse array is fabricated using cost-effective, highly stable, flexible, and eco-friendly carbon materials, offering efficient analog multiply-accumulate operations. To illustrate the complexity of the proposed brain-inspired hierarchical interactive IMC system, three modules are proposed: 1) unimodal extraction module, 2) hierarchical interactive module, 3) output module. Furthermore, the proposed system is validated by applying it to video sentiment analysis. The experimental results demonstrate that the proposed system outperforms the existing state-of-the-art methods with high computational efficiency and good robustness. This work opens up a new way to achieve the deep integration of nanomaterials, deep learning, and modern electronics into IMC.

MMFIL-Net: Multi-level and multi-source feature interactive lightweight network for polyp segmentation

Accuracy, generalized performance, and model size are critical considerations for designing a real-time polyp segmentation model. However, existing techniques primarily focus on accuracy and do not consider the other two vital characteristics. This paper proposes MMFIL-Net, a novel polyp segmentation technique. As part of MMFIL-Net, the Hierarchical Multi-source Feature Interaction Module (HMFIM) comprises Multi-source Feature Interaction Blocks (MFIB). MFIB manipulates multi-level and multi-sourced features to reduce the gap between low and high-level feature maps to achieve generalized performance. Additionally, the Multiple Receptive Field Feature Interaction Block (MRFFIB) targets the issues of segmenting polyps of different sizes. Finally, Dual Source Attention Fusion Block (DSAFB) is introduced to deal with hazy boundary information for early-stage polyps detection and segmentation. The proposed model outperformed existing lightweight models in conducted evaluation on different datasets. In addition to the achieved generalized performance and higher accuracy, the proposed model presents a significant reduction in the model size than existing approaches. The proposed model only contains 6.68 million parameters and has 4.32G MACs (Multiply–Accumulate Operations), which is better than the current approaches.

RCKD: Response-Based Cross-Task Knowledge Distillation for Pathological Image Analysis.

We propose a novel transfer learning framework for pathological image analysis, the Response-based Cross-task Knowledge Distillation (RCKD), which improves the performance of the model by pretraining it on a large unlabeled dataset guided by a high-performance teacher model. RCKD first pretrains a student model to predict the nuclei segmentation results of the teacher model for unlabeled pathological images, and then fine-tunes the pretrained model for the downstream tasks, such as organ cancer sub-type classification and cancer region segmentation, using relatively small target datasets. Unlike conventional knowledge distillation, RCKD does not require that the target tasks of the teacher and student models be the same. Moreover, unlike conventional transfer learning, RCKD can transfer knowledge between models with different architectures. In addition, we propose a lightweight architecture, the Convolutional neural network with Spatial Attention by Transformers (CSAT), for processing high-resolution pathological images with limited memory and computation. CSAT exhibited a top-1 accuracy of 78.6% on ImageNet with only 3M parameters and 1.08 G multiply-accumulate (MAC) operations. When pretrained by RCKD, CSAT exhibited average classification and segmentation accuracies of 94.2% and 0.673 mIoU on six pathological image datasets, which is 4% and 0.043 mIoU higher than EfficientNet-B0, and 7.4% and 0.006 mIoU higher than ConvNextV2-Atto pretrained on ImageNet, respectively.

Multi-modal medical Transformers: A meta-analysis for medical image segmentation in oncology

Multi-modal medical image segmentation is a crucial task in oncology that enables the precise localization and quantification of tumors. The aim of this work is to present a meta-analysis of the use of multi-modal medical Transformers for medical image segmentation in oncology, specifically focusing on multi-parametric MR brain tumor segmentation (BraTS2021), and head and neck tumor segmentation using PET-CT images (HECKTOR2021). The multi-modal medical Transformer architectures presented in this work exploit the idea of modality interaction schemes based on visio-linguistic representations: (i) single-stream, where modalities are jointly processed by one Transformer encoder, and (ii) multiple-stream, where the inputs are encoded separately before being jointly modeled. A total of fourteen multi-modal architectures are evaluated using different ranking strategies based on dice similarity coefficient (DSC) and average symmetric surface distance (ASSD) metrics. In addition, cost indicators such as the number of trainable parameters and the number of multiply-accumulate operations (MACs) are reported. The results demonstrate that multi-path hybrid CNN-Transformer-based models improve segmentation accuracy when compared to traditional methods, but come at the cost of increased computation time and potentially larger model size.

Laterally gated ferroelectric field effect transistor (LG-FeFET) using α-In2Se3 for stacked in-memory computing array

In-memory computing is an attractive alternative for handling data-intensive tasks as it employs parallel processing without the need for data transfer. Nevertheless, it necessitates a high-density memory array to effectively manage large data volumes. Here, we present a stacked ferroelectric memory array comprised of laterally gated ferroelectric field-effect transistors (LG-FeFETs). The interlocking effect of the α-In2Se3 is utilized to regulate the channel conductance. Our study examined the distinctive characteristics of the LG-FeFET, such as a notably wide memory window, effective ferroelectric switching, long retention time (over 3 × 104 seconds), and high endurance (over 105 cycles). This device is also well-suited for implementing vertically stacked structures because decreasing its height can help mitigate the challenges associated with the integration process. We devised a 3D stacked structure using the LG-FeFET and verified its feasibility by performing multiply-accumulate (MAC) operations in a two-tier stacked memory configuration.

Lightweight Seizure Detection Based on Multi-Scale Channel Attention.

Epilepsy is one kind of neurological disease characterized by recurring seizures. Recurrent seizures can cause ongoing negative mental and cognitive damage to the patient. Therefore, timely diagnosis and treatment of epilepsy are crucial for patients. Manual electroencephalography (EEG) signals analysis is time and energy consuming, making automatic detection using EEG signals particularly important. Many deep learning algorithms have thus been proposed to detect seizures. These methods rely on expensive and bulky hardware, which makes them unsuitable for deployment on devices with limited resources due to their high demands on computer resources. In this paper, we propose a novel lightweight neural network for seizure detection using pure convolutions, which is composed of inverted residual structure and multi-scale channel attention mechanism. Compared with other methods, our approach significantly reduces the computational complexity, making it possible to deploy on low-cost portable devices for seizures detection. We conduct experiments on the CHB-MIT dataset and achieves 98.7% accuracy, 98.3% sensitivity and 99.1% specificity with 2.68[Formula: see text]M multiply-accumulate operations (MACs) and only 88[Formula: see text]K parameters.

SMART: Selective MAC zero-optimization for neural network reliability under radiation

Neural networks running on low-power edge devices can help in achieving ubiquitous computing with limited infrastructure. When such edge devices are deployed in conventional and extreme environments without the necessary shielding, they must be fault tolerant for reliable operation. As a pilot study, we focused on embedding fault tolerance into neural networks by proposing a novel selective multiply-accumulate zero-optimization technique based on whether the value of an input provided to a neuron of a neural network is zero. If the value is zero, then the corresponding multiply-accumulate operation is bypassed. We subjected the operating system-based implementation of the optimization technique to radiation test campaigns using approximately 14 MeV neutrons, and found the proposed optimization technique to improve the fault tolerance of the tested neural network by approximately 44%.

An Energy-Efficient Bayesian Neural Network Accelerator With CiM and a Time-Interleaved Hadamard Digital GRNG Using 22-nm FinFET

Bayesian neural networks (BNNs) have been proposed to address the problems of overfitting and overconfident decision making, common in conventional neural networks (NNs), due to their ability to model and express uncertainty in their predictions. However, BNNs require multiple inference passes to produce the necessary posterior distributions used to generate these highly desirable uncertainty estimates. As such, BNNs require not only an efficient, high-performance multiply-accumulation (MAC) operation but also an efficient Gaussian random number generator (GRNG) with high-quality statistics. In this article, an NN accelerator chip, leveraging a multi-bit analog compute-in-memory (CiM) static random-access memory (SRAM) macro, with a tightly coupled and highly efficient GRNG scheme, is presented in the Intel 22FFL process. The CiM macro achieves a peak energy efficiency of 32.2 TOP/sW, with 8-bit precision, while ensuring accurate on-chip matrix-vector multiplications (MVMs) with a computation error less than 0.5%. The variable precision GRNG achieves a peak throughput of 7.31 GSamp/s for an energy efficiency of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim$</tex-math> </inline-formula> 1 TSamp/J. Overall, our proposed system achieves a peak energy efficiency of 1170 GOP/s/W, a 35–133 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> improvement over the state-of-the-art BNN accelerators, with 98.14% accuracy for the MNIST dataset.

DeepFire2: A Convolutional Spiking Neural Network Accelerator on FPGAs

Brain-inspired spiking neural networks (SNNs) replace the multiply-accumulate operations of traditional neural networks by integrate-and-fire neurons, with the goal of achieving greater energy efficiency. Specialized hardware implementations of those neurons clearly have advantages over general-purpose devices in terms of power and performance, but exhibit poor scalability when it comes to accelerating large neural networks. DeepFire2 introduces a hardware architecture which can map large network layers efficiently across multiple super logic regions in a multi-die FPGA. That gives more control over resource allocation and parallelism, benefiting both throughput and energy consumption. Avoiding the use of lookup tables to implement the <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AND</monospace> operations of an SNN, prevents the layer size to be limited by logic resources. A deep pipeline does not only lead to an increased clock speed of up to 600 MHz. We double the throughput and power efficiency compared to our previous version of DeepFire, which equates to an almost 10-fold improvement over other previous implementations. Importantly, we are able to deploy a large ImageNet model, while maintaining a throughput of over 1500 frames per second.

Multitask Deep Learning for Human Activity, Speed, and Body Weight Estimation Using Commercial Smart Insoles

Healthcare professionals and individual users use wearable devices equipped with various sensors for healthcare management. Recently, the joint usage of artificial intelligence and these wearable sensors has played an essential role in healthcare management by providing a wide range of applications such as fitness tracking, gym activity monitoring, patient rehabilitation monitoring, and disease detection. These tasks eventually aim to enhance personal well-being and better manage the user’s physical health by monitoring different activity types and body weight changes. Here, we present an efficient multi-task learning framework based on commercial smart insoles that can solve three tasks related to physical health management: activity classification, speed estimation, and body weight estimation. Our multi-task framework converts the sensor data from the smart insole to a recurrence plot, which shows significant performance improvement compared to processing the raw time series data. In addition, we utilized a modified MobileNetV2 as our backbone network, which has a total parameter of less than 100K and a computational budget of 0.34G of multiply-accumulate operations. Furthermore, we collected a vast dataset from 72 users carrying out 16 experiments, which contains the largest number of people for multi-task learning purposes using smart insoles. Extensive experiments show that the proposed multi-task learning framework is extremely efficient while outperforming or leading to comparable performance against single-task models.

Analog Multiply-Accumulate Cell With Multi-Bit Resolution for All-Analog AI Inference Accelerators

Mixed-signal AI accelerators offer the possibility of higher energy efficiency for moderate resolution computations compared to their digital counterparts. All-analog implementations, where all operations are performed in the analog domain, can further improve this energy advantage. An energy efficient multiply-accumulate cell for all-analog neural layer processing macros is presented. The proposed analog two-quadrant multiplier circuit consists of two complementary MOSFETs where the pulse width modulated input activation is applied to the gates and the weight signal to the isolated back-gate. The analog multi-bit resolution weight is dynamically stored on a memory capacitor. The multiply-accumulate operation result is represented by charge accumulated on a summation line and drawn from or put onto a computation capacitance. Simulation results based on a 22 nm FD-SOI CMOS technology show that the cell consumes about 0.67 fJ for a circuit-level multiply-accumulate operation. An area efficiency of 166 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> 10 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^{12}$</tex-math> </inline-formula> MAC/s/mm <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^2$</tex-math> </inline-formula> is achieved.

FlexBlock: A Flexible DNN Training Accelerator With Multi-Mode Block Floating Point Support

When training deep neural networks (DNNs), expensive floating point arithmetic units are used in GPUs or custom neural processing units (NPUs). To reduce the burden of floating point arithmetic, community has started exploring the use of more efficient data representations, e.g., block floating point (BFP). The BFP format allows a group of values to share an exponent, which effectively reduces the memory footprint and enables cheaper fixed point arithmetic for multiply-accumulate (MAC) operations. However, existing BFP-based DNN accelerators are targeted for a specific precision, making them less versatile. In this paper, we present FlexBlock, a DNN training accelerator with three BFP modes, possibly different among activation, weight, and gradient tensors. By configuring FlexBlock to a lower BFP precision, the number of MACs handled by the core increases by up to 4× in 8-bit mode or 16× in 4-bit mode compared to 16-bit mode. To reach this theoretical upper bound, FlexBlock maximizes the core utilization at various precision levels or layer types, and allows dynamic precision control to keep throughput at its peak without sacrificing training accuracy. We evaluate the effectiveness of FlexBlock using representative DNNs on CIFAR, ImageNet and WMT14 datasets. As a result, training in FlexBlock significantly improves training speed by 1.5 5.3× and energy efficiency by 2.4 7.0× compared to other training accelerators.