Machine Learning Accelerators Research Articles

Survey of Security Issues in Memristor-Based Machine Learning Accelerators for RF Analysis

Survey of Security Issues in Memristor-Based Machine Learning Accelerators for RF Analysis

Brain-machine interactive neuromodulation research tool with edge AI computing

Closed-loop neuromodulation with intelligence methods has shown great potentials in providing novel neuro-technology for treating neurological and psychiatric diseases. Development of brain-machine interactive neuromodulation strategies could lead to breakthroughs in precision and personalized electronic medicine. The neuromodulation research tool integrating artificial intelligent computing and performing neural sensing and stimulation in real-time could accelerate the development of closed-loop neuromodulation strategies and translational research into clinical application. In this study, we developed a brain-machine interactive neuromodulation research tool (BMINT), which has capabilities of neurophysiological signals sensing, computing with mainstream machine learning algorithms and delivering electrical stimulation pulse by pulse in real-time. The BMINT research tool achieved system time delay under 3 ms, and computing capabilities in feasible computation cost, efficient deployment of machine learning algorithms and acceleration process. Intelligent computing framework embedded in the BMINT enable real-time closed-loop neuromodulation developed with mainstream AI ecosystem resources. The BMINT could provide timely contribution to accelerate the translational research of intelligent neuromodulation by integrating neural sensing, edge AI computing and stimulation with AI ecosystems.

Bridging the gap between machine learning and particle accelerator physics with high-speed, differentiable simulations

Machine learning has emerged as a powerful solution to the modern challenges in accelerator physics. However, the limited availability of beam time, the computational cost of simulations, and the high dimensionality of optimization problems pose significant challenges in generating the required data for training state-of-the-art machine learning models. In this work, we introduce heetah, a yorch-based high-speed differentiable linear beam dynamics code. heetah enables the fast collection of large datasets by reducing computation times by multiple orders of magnitude and facilitates efficient gradient-based optimization for accelerator tuning and system identification. This positions heetah as a user-friendly, readily extensible tool that integrates seamlessly with widely adopted machine learning tools. We showcase the utility of heetah through five examples, including reinforcement learning training, gradient-based beamline tuning, gradient-based system identification, physics-informed Bayesian optimization priors, and modular neural network surrogate modeling of space charge effects. The use of such a high-speed differentiable simulation code will simplify the development of machine learning-based methods for particle accelerators and fast-track their integration into everyday operations of accelerator facilities. Published by the American Physical Society 2024

Cerberus: Triple Mode Acceleration of Sparse Matrix and Vector Multiplication

The multiplication of sparse matrix and vector (SpMV) is one of the most widely used kernels in high-performance computing as well as machine learning acceleration for sparse neural networks. The design space of SpMV accelerators has two axes: algorithm and matrix representation. There have been two widely used algorithms and data representations. Two algorithms, scalar multiplication and dot product, can be combined with two sparse data representations, compressed sparse and bitmap formats for the matrix and vector. Although the prior accelerators adopted one of the possible designs, it is yet to be investigated which design is the best one across different hardware resources and workload characteristics. This paper first investigates the impact of design choices with respect to the algorithm and data representation. Our evaluation shows that no single design always outperforms the others across different workloads, but the two best designs (i.e., compressed sparse format and bitmap format with dot product) have complementary performance with trade-offs incurred by the matrix characteristics. Based on the analysis, this study proposes Cerberus, a triple-mode accelerator supporting two sparse operation modes in addition to the base dense mode. To allow such multi-mode operation, it proposes a prediction model based on matrix characteristics under a given hardware configuration, which statically selects the best mode for a given sparse matrix with its dimension and density information. Our experimental results show that Cerberus provides 12.1× performance improvements from a dense-only accelerator, and 1.5× improvements from a fixed best SpMV design.

An Open-Source ML-Based Full-Stack Optimization Framework for Machine Learning Accelerators

Parameterizable machine learning (ML) accelerators are the product of recent breakthroughs in ML. To fully enable their design space exploration (DSE), we propose a physical-design-driven, learning-based prediction framework for hardware-accelerated deep neural network (DNN) and non-DNN ML algorithms. It adopts a unified approach that combines power, performance, and area (PPA) analysis with frontend performance simulation, thereby achieving a realistic estimation of both backend PPA and system metrics such as runtime and energy. In addition, our framework includes a fully automated DSE technique, which optimizes backend and system metrics through an automated search of architectural and backend parameters. Experimental studies show that our approach consistently predicts backend PPA and system metrics with an average 7% or less prediction error for the ASIC implementation of two deep learning accelerator platforms, VTA and VeriGOOD-ML, in both a commercial 12 nm process and a research-oriented 45 nm process.

Ferroelectric capacitors and field-effect transistors as in-memory computing elements for machine learning workloads

This study discusses the feasibility of Ferroelectric Capacitors (FeCaps) and Ferroelectric Field-Effect Transistors (FeFETs) as In-Memory Computing (IMC) elements to accelerate machine learning (ML) workloads. We conducted an exploration of device fabrication and proposed system-algorithm co-design to boost performance. A novel FeCap device, incorporating an interfacial layer (IL) and Hf0.5Zr0.5O2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ ext {Hf}_{0.5}\ ext {Zr}_{0.5}\ ext {O}_2$$\\end{document} (HZO), ensures a reduction in operating voltage and enhances HZO scaling while being compatible with CMOS circuits. The IL also enriches ferroelectricity and retention properties. When integrated into crossbar arrays, FeCaps and FeFETs demonstrate their effectiveness as IMC components, eliminating sneak paths and enabling selector-less operation, leading to notable improvements in energy efficiency and area utilization. However, it is worth noting that limited capacitance ratios in FeCaps introduced errors in multiply-and-accumulate (MAC) computations. The proposed co-design approach helps in mitigating these errors and achieves high accuracy in classifying the CIFAR-10 dataset, elevating it from a baseline of 10% to 81.7%. FeFETs in crossbars, with a higher on-off ratio, outperform FeCaps, and our proposed charge-based sensing scheme achieved at least an order of magnitude reduction in power consumption, compared to prevalent current-based methods.

M3ICRO: Machine learning-enabled compact photonic tensor core based on programmable multi-operand multimode interference

Photonic computing shows promise for transformative advancements in machine learning (ML) acceleration, offering ultrafast speed, massive parallelism, and high energy efficiency. However, current photonic tensor core (PTC) designs based on standard optical components hinder scalability and compute density due to their large spatial footprint. To address this, we propose an ultracompact PTC using customized programmable multi-operand multimode interference (MOMMI) devices, named M3ICRO. The programmable MOMMI leverages the intrinsic light propagation principle, providing a single-device programmable matrix unit beyond the conventional computing paradigm of one multiply-accumulate operation per device. To overcome the optimization difficulty of customized devices that often requires time-consuming simulation, we apply ML for optics to predict the device behavior and enable differentiable optimization flow. We thoroughly investigate the reconfigurability and matrix expressivity of our customized PTC and introduce a novel block unfolding method to fully exploit the computing capabilities of a complex-valued PTC for near-universal real-valued linear transformations. Extensive evaluations demonstrate that M3ICRO achieves a 3.5–8.9× smaller footprint, 1.6–4.4× higher speed, 9.9–38.5× higher compute density, 3.7–12× higher system throughput, and superior noise robustness compared to state-of-the-art coherent PTC designs. It also outperforms electronic digital A100 graphics processing unit by 34.8–403× higher throughput while maintaining close-to-digital task accuracy across various ML benchmarks.

Privacy‐preserving task offloading in mobile edge computing: A deep reinforcement learning approach

AbstractAs machine learning (ML) technologies continue to evolve, there is an increasing demand for data. Mobile crowd sensing (MCS) can motivate more users in the data collection process through reasonable compensation, which can enrich the data scale and coverage. However, nowadays, users are increasingly concerned about their privacy and are unwilling to easily share their personal data. Therefore, protecting privacy has become a crucial issue. In ML, federated learning (FL) is a widely known privacy‐preserving technique where the model training process is performed locally by the data owner, which can protect privacy to a large extent. However, as the model size grows, the weak computing power and battery life of user devices are not sufficient to support training a large number of models locally. With mobile edge computing (MEC), user can offload some of the model training tasks to the edge server for collaborative computation, allowing the edge server to participate in the model training process to improve training efficiency. However, edge servers are not fully trusted, and there is still a risk of privacy leakage if data is directly uploaded to the edge server. To address this issue, we design a local differential privacy (LDP) based data privacy‐preserving algorithm and a deep reinforcement learning (DRL) based task offloading algorithm. We also propose a privacy‐preserving distributed ML framework for MEC and model the cloud‐edge‐mobile collaborative training process. These algorithms not only enable effective utilization of edge computing to accelerate machine learning model training but also significantly enhance user privacy and save device battery power. We have conducted experiments to verify the effectiveness of the framework and algorithms.

Dread and the automation of education: From algorithmic anxiety to a new sensibility

Accelerating digitization, algorithmic computation, artificial intelligence, and machine learning, along with the increasing automation of work, communication, and everyday life, are central to critical studies of technology and political economy, as well as to public discourse concerning technology’s role in creating futures. Ongoing transformations in technological capacity have also been scrutinized for their impact on experience, emotion, and culture. Building on David Theo Goldberg’s assertion that “tracking capitalism” creates pervasive dread, this article explores how the digital automation of education, specifically, generates forms of algorithmic anxiety that impact teaching and learning, constraining pedagogical visions of alternative futures. Algorithmic anxiety in education contributes to dread’s proliferation. If dread is the “driving social sensibility” today, and algorithmic anxiety a pedagogical vector of its spread, then a less dreadful education should center an imaginative struggle for new sensibilities.

Factored Systolic Arrays Based on Radix-8 Multiplication for Machine Learning Acceleration

Factored Systolic Arrays Based on Radix-8 Multiplication for Machine Learning Acceleration

VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations

Processing-in-Memory (PIM) has been widely explored for accelerating data-intensive machine learning computation that mainly consists of general-matrix-multiplication (GEMM), by mitigating the burden of data movements and exploiting the ultra-high memory parallelism. The two mainstreams of PIM, the analog- and digital-type, have both been exploited in accelerating machine learning workloads by numerous outstanding prior works. Currently, the digital-PIM is increasingly favored due to the broader computing support and the avoidance of errors caused by intrinsic non-idealities, e.g., process variation. Nevertheless, it still lacks further optimization considering the characteristics of the GEMM computation, including better efficient data layout and scheduling, and the ability to handle the sparsity of activations at the bit-level. To boost the performance and efficiency of digital SRAM PIM, we propose the architecture called VSPIM that performs the computation in a bit-serial fashion, with unique support of vector-scalar computing pattern. The novelties of the VSPIM can be concluded as follows: 1) support bit-serial based scalar-vector computing via ingenious parallel bit-broadcasting; 2) refine the GEMM mapping strategy and computing pattern to enhance performance and efficiency; 3) powered by the introduced scalar-vector operation, the bit-sparsity of activation is leveraged to halt unnecessary computation to maximize efficiency and throughput. Our comprehensive evaluation shows that, compared to the state-of-the-art SRAM-based digital-PIM design (Neural Cache), VSPIM can significantly boost the performance and energy efficiency by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$8.87\times$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$4.81\times$</tex-math></inline-formula> respectively, with negligible area overhead, upon multiple representative neural networks.

Energy efficiency in edge TPU vs. embedded GPU for computer-aided medical imaging segmentation and classification

In this work, we evaluate the energy usage of fully embedded medical diagnosis aids based on both segmentation and classification of medical images implemented on Edge TPU and embedded GPU processors. We use glaucoma diagnosis based on color fundus images as an example to show the possibility of performing segmentation and classification in real time on embedded boards and to highlight the different energy requirements of the studied implementations.Several other works develop the use of segmentation and feature extraction techniques to detect glaucoma, among many other pathologies, with deep neural networks. Memory limitations and low processing capabilities of embedded accelerated systems (EAS) limit their use for deep network-based system training. However, including specific acceleration hardware, such as NVIDIA's Maxwell GPU or Google's Edge TPU, enables them to perform inferences using complex pre-trained networks in very reasonable times.In this study, we evaluate the timing and energy performance of two EAS equipped with Machine Learning (ML) accelerators executing an example diagnostic tool developed in a previous work. For optic disc (OD) and cup (OC) segmentation, the obtained prediction times per image are under 29 and 43 ms using Edge TPUs and Maxwell GPUs, respectively. Prediction times for the classification subsystem are lower than 10 and 14 ms for Edge TPUs and Maxwell GPUs, respectively. Regarding energy usage, in approximate terms, for OD segmentation Edge TPUs and Maxwell GPUs use 38 and 190 mJ per image, respectively. For fundus classification, Edge TPUs and Maxwell GPUs use 45 and 70 mJ, respectively.

Accelerating Machine Learning Inference with GPUs in ProtoDUNE Data Processing

We study the performance of a cloud-based GPU-accelerated inference server to speed up event reconstruction in neutrino data batch jobs. Using detector data from the ProtoDUNE experiment and employing the standard DUNE grid job submission tools, we attempt to reprocess the data by running several thousand concurrent grid jobs, a rate we expect to be typical of current and future neutrino physics experiments. We process most of the dataset with the GPU version of our processing algorithm and the remainder with the CPU version for timing comparisons. We find that a 100-GPU cloud-based server is able to easily meet the processing demand, and that using the GPU version of the event processing algorithm is two times faster than processing these data with the CPU version when comparing to the newest CPUs in our sample. The amount of data transferred to the inference server during the GPU runs can overwhelm even the highest-bandwidth network switches, however, unless care is taken to observe network facility limits or otherwise distribute the jobs to multiple sites. We discuss the lessons learned from this processing campaign and several avenues for future improvements.

Instruction-Level Power Side-Channel Leakage Evaluation of Soft-Core CPUs on Shared FPGAs

Side-channel disassembly attacks recover CPU instructions from power or electromagnetic side-channel traces measured during code execution. These attacks typically rely on physical access, proximity to the victim device, and high sampling rate measuring instruments. In this work, however, we analyze the CPU instruction-level power side-channel leakage in an environment that lacks physical access or expensive measuring equipment. We show that instruction leakage is present even in a multitenant FPGA scenario, where the victim uses a soft-core CPU, and the adversary deploys on-chip voltage-fluctuation sensors. Unlike previous remote power side-channel attacks, which either require a considerable number of victim traces or attack large victim circuits such as machine learning accelerators, we take an evaluator’s point of view and provide an analysis of the instruction-level power side-channel leakage of a small open-source RISC-V soft processor core. To investigate whether the power side-channel traces leak secrets, we profile the victim device and implement various instruction opcode classifiers based on both classical machine learning algorithms used in disassembly attacks, and novel, deep learning approaches. We explore how parameters such as placement, trace averaging, profiling templates, and different FPGA families (including a cloud-scale FPGA) impact the classification accuracy. Despite the limited leakage of the soft-core CPU victim and a reduced accuracy and sampling rate of on-chip sensors, we show that in a worst-case scenario for the evaluator, i.e., an attacker breaching physical separation, we can identify the opcode of executed instructions with an average accuracy as high as 86.46%. Our analysis shows that determining the executed instruction type is not a classification bottleneck, while leakages between instructions of the same type can be challenging for deep learning models to distinguish. We also show that the instruction-level leakage is significantly reduced in a cloud-scale FPGA scenario with higher soft-core CPU frequencies. Nevertheless, our results show that even small circuits, such as soft-core CPUs, leak potentially exploitable information through on-chip power side channels, and users should deploy mitigation techniques against disassembly attacks to protect their proprietary code and data.

SAMBA: Sparsity Aware In-Memory Computing Based Machine Learning Accelerator

Machine Learning (ML) inference is typically dominated by highly data-intensive Matrix Vector Multiplication (MVM) computations that may be constrained by the memory bottleneck due to massive data movement between processor and memory units. Although analog in-memory computing (IMC) ML accelerators have been proposed to execute MVM with high efficiency, the latency and energy of such analog computing systems can be dominated by the large latency and energy costs from analog-to-digital converters (ADC). Leveraging sparsity in ML workloads, reconfigurable ADCs can improve the MVM energy and latency by reducing the required ADC bit precision. However, such improvement in latency can be hindered by non-uniform sparsity of the weight matrices mapped into hardware. Moreover, data movement between MVM processing cores may become another factor that delays the overall system-level performance. To address these issues, we propose SAMBA, Sparsity Aware IMC Based Machine Learning Accelerator. First, we propose load balancing at the mapping of weight matrices into physical crossbars. The goal of load balancing is to accommodate reconfigurable ADCs and address the possible delay of MVM processing caused by non-uniform sparsity in the weight matrices. Second, we propose optimizations in arranging and scheduling the tiled MVM hardware to minimize the overhead of data movement across multiple processing cores. Our evaluations show that the proposed load balancing technique can achieve performance improvement by eliminating the non-uniformity in the sparsity of mapped matrices. Moreover, we demonstrate that data movement optimizations can further improve both performance and energy efficiency regardless of sparsity condition. With the combination of load balancing and data movement optimization in conjunction with reconfigurable ADCs, our proposed approach provides up to 2.38x speed-up and 1.54x energy efficiency over state-of-art analog IMC based ML accelerators for ImageNet datasets on Resnet-50 architecture.

FPGA-Based Machine Learning: Platforms, Applications, Design Considerations, Challenges, and Future Directions

Field-Programmable Gate Arrays (FPGAs) have emerged as a promising platform for accelerating machine learning tasks due to their high parallelism, low latency, and hardware customization ability. In this paper, the authors provide an overview of popular FPGA platforms for machine learning and compare the tradeoffs among FPGAs, GPUs, and CPUs for machine learning. The authors also present specific applications of machine learning based on FPGAs, including those in autonomous driving and healthcare. Additionally, the paper explores FPGA design considerations, such as architecture, resource utilization, and power consumption. Nonetheless, obstacles persist in the realm of FPGA-based machine learning that require attention. Identifying the ideal balance between adaptability and performance, considering factors such as space, energy usage, and latency, is still challenging. As the capabilities of FPGAs expand, there is a significant need for devices that have a smaller footprint, reduced power consumption, and minimized delays. The paper emphasizes the necessity of ongoing research in the field of FPGA-based machine learning to address these issues and continue enhancing the performance and effectiveness of machine learning systems.

Bottom-Up and Top-Down Approaches for the Design of Neuromorphic Processing Systems: Tradeoffs and Synergies Between Natural and Artificial Intelligence

While Moore's law has driven exponential computing power expectations, its nearing end calls for new avenues for improving the overall system performance. One of these avenues is the exploration of alternative brain-inspired computing architectures that aim at achieving the flexibility and computational efficiency of biological neural processing systems. Within this context, neuromorphic engineering represents a paradigm shift in computing based on the implementation of spiking neural network architectures in which processing and memory are tightly co-located. In this paper, we provide a comprehensive overview of the field, highlighting the different levels of granularity at which this paradigm shift is realized and comparing design approaches that focus on replicating natural intelligence (bottom-up) versus those that aim at solving practical artificial intelligence applications (top-down). First, we present the analog, mixed-signal and digital circuit design styles, identifying the boundary between processing and memory through time multiplexing, in-memory computation, and novel devices. Then, we highlight the key tradeoffs for each of the bottom-up and top-down design approaches, survey their silicon implementations, and carry out detailed comparative analyses to extract design guidelines. Finally, we identify necessary synergies and missing elements required to achieve a competitive advantage for neuromorphic systems over conventional machine-learning accelerators in edge computing applications, and outline the key ingredients for a framework toward neuromorphic intelligence.

Hashing for secure optical information compression in a heterogeneous convolutional neural network

In recent years, heterogeneous machine learning accelerators have become of significant interest to science, engineering, and industry. At the same time, the looming post-quantum encryption era instigates the demand for increased data security. From a hardware processing point of view, electronic computing hardware is challenged by electronic capacitive interconnect delay and associated energy consumption. In heterogeneous systems, such as electronic–photonic accelerators, parasitic domain crossings limit throughput and speed. With analog optical accelerators exhibiting a strong potential for high throughput (up to petaoperations per second) and operation efficiency, their ability to perform machine learning classification tasks on encrypted data has not been broadly recognized. This work is a significant step in that direction. Here, we present an optical hashing and compression scheme that is inspired by SWIFFT, a post-quantum hashing family of algorithms. High degree optical hardware-to-algorithm homomorphism allows one to optimally harvest the potential of free-space data processing: innate parallelism, low latency tensor by-element multiplication, and zero-energy Fourier transformation operations. The algorithm can provide several orders of magnitude increase in processing speed as compared to optical machine learning accelerators with non-compressed input. This is achieved by replacing slow, high-resolution CMOS cameras with ultra-fast and signal-triggered CMOS detector arrays. Additionally, information acquired in this way will require much lower transmission throughput, less in silico processing power, storage, and will be pre-hashed, facilitating optical information security. This concept has the potential to allow heterogeneous convolutional Fourier classifiers to approach the performance of their fully electronic counterparts and enables data classification on hashed data.

Power-Intent Systolic Array Using Modified Parallel Multiplier for Machine Learning Acceleration.

Systolic arrays are an integral part of many modern machine learning (ML) accelerators due to their efficiency in performing matrix multiplication that is a key primitive in modern ML models. Current state-of-the-art in systolic array-based accelerators mainly target area and delay optimizations with power optimization being considered as a secondary target. Very few accelerator designs directly target power optimizations and that too using very complex algorithmic modifications that in turn result in a compromise in the area or delay performance. We present a novel Power-Intent Systolic Array (PI-SA) that is based on the fine-grained power gating of the multiplication and accumulation (MAC) block multiplier inside the processing element of the systolic array, which reduces the design power consumption quite significantly, but with an additional delay cost. To offset the delay cost, we introduce a modified decomposition multiplier to obtain smaller reduction tree and to further improve area and delay, we also replace the carry propagation adder with a carry save adder inside each sub-multiplier. Comparison of the proposed design with the baseline Gemmini naive systolic array design and its variant, i.e., a conventional systolic array design, exhibits a delay reduction of up to 6%, an area improvement of up to 32% and a power reduction of up to 57% for varying accumulator bit-widths.

CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework

Recently, automated co-design of machine learning (ML) models and accelerator architectures has attracted significant attention from both the industry and academia. However, most co-design frameworks either explore a limited search space or employ suboptimal exploration techniques for simultaneous design decision investigations of the ML model and the accelerator. Furthermore, training the ML model and simulating the accelerator performance is computationally expensive. To address these limitations, this work proposes a novel neural architecture and hardware accelerator co-design framework, called CODEBench. It comprises two new benchmarking sub-frameworks, CNNBench and AccelBench, which explore expanded design spaces of convolutional neural networks (CNNs) and CNN accelerators. CNNBench leverages an advanced search technique, Bayesian Optimization using Second-order Gradients and Heteroscedastic Surrogate Model for Neural Architecture Search, to efficiently train a neural heteroscedastic surrogate model to converge to an optimal CNN architecture by employing second-order gradients. AccelBench performs cycle-accurate simulations for diverse accelerator architectures in a vast design space. With the proposed co-design method, called Bayesian Optimization using Second-order Gradients and Heteroscedastic Surrogate Model for Co-Design of CNNs and Accelerators, our best CNN–accelerator pair achieves 1.4% higher accuracy on the CIFAR-10 dataset compared to the state-of-the-art pair while enabling 59.1% lower latency and 60.8% lower energy consumption. On the ImageNet dataset, it achieves 3.7% higher Top1 accuracy at 43.8% lower latency and 11.2% lower energy consumption. CODEBench outperforms the state-of-the-art framework, i.e., Auto-NBA, by achieving 1.5% higher accuracy and 34.7× higher throughput while enabling 11.0× lower energy-delay product and 4.0× lower chip area on CIFAR-10.