Machine Learning Accelerators Research Articles

Method for Detecting Roadbed Compaction Degree Based on Machine Learning and Vibration Acceleration

Roadbed construction occupies a core position in highway construction, but its quality is easily constrained by multiple factors such as changing environmental factors, the performance of construction equipment, and the professional abilities of construction personnel, leading to potential quality risks. Traditional quality inspection methods are mostly carried out after construction is completed, making it difficult to achieve continuous and real-time monitoring of roadbed compaction quality, which to some extent limits the real-time feedback and adjustment of construction quality. Vibration compaction technology has been widely used in the field of highway engineering due to its high efficiency and speed. The compaction degree is directly related to the durability and service life of the highway; therefore, accurate and efficient detection of compaction degree is crucial. This article proposes a method for detecting roadbed compaction degree by integrating machine learning (ML) and vibration acceleration signals. This method aims to achieve accurate evaluation of roadbed compaction by real-time monitoring and analysis of vibration acceleration data, combined with the powerful prediction and classification capabilities of ML algorithms. The experimental results show that this method not only improves the detection efficiency, but also significantly enhances the accuracy of compaction degree detection.

VSPIM: SRAM Processing-in-Memory DNN Acceleration via Vector-Scalar Operations

Processing-in-Memory (PIM) has been widely explored for accelerating data-intensive machine learning computation that mainly consists of general-matrix-multiplication (GEMM), by mitigating the burden of data movements and exploiting the ultra-high memory parallelism. The two mainstreams of PIM, the analog- and digital-type, have both been exploited in accelerating machine learning workloads by numerous outstanding prior works. Currently, the digital-PIM is increasingly favored due to the broader computing support and the avoidance of errors caused by intrinsic non-idealities, e.g., process variation. Nevertheless, it still lacks further optimization considering the characteristics of the GEMM computation, including better efficient data layout and scheduling, and the ability to handle the sparsity of activations at the bit-level. To boost the performance and efficiency of digital SRAM PIM, we propose the architecture called VSPIM that performs the computation in a bit-serial fashion, with unique support of vector-scalar computing pattern. The novelties of the VSPIM can be concluded as follows: 1) support bit-serial based scalar-vector computing via ingenious parallel bit-broadcasting; 2) refine the GEMM mapping strategy and computing pattern to enhance performance and efficiency; 3) powered by the introduced scalar-vector operation, the bit-sparsity of activation is leveraged to halt unnecessary computation to maximize efficiency and throughput. Our comprehensive evaluation shows that, compared to the state-of-the-art SRAM-based digital-PIM design (Neural Cache), VSPIM can significantly boost the performance and energy efficiency by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$8.87\times$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$4.81\times$</tex-math></inline-formula> respectively, with negligible area overhead, upon multiple representative neural networks.

Quantum Machine Learning: Leveraging Quantum Computing for Enhanced Learning Algorithms

The paper "Quantum Machine Learning: Leveraging Quantum Computing for Enhanced Learning Algorithms" explores the integration of quantum computing principles into classical machine learning techniques, aiming to address limitations such as scalability and computational inefficiency. It presents the foundational concepts of quantum computing, including superposition and entanglement, and their application in accelerating machine learning processes. The study emphasizes the potential for quantum algorithms to significantly improve the performance of machine learning tasks by processing large datasets more efficiently and exploring larger hypothesis spaces. Key quantum machine learning algorithms discussed include Quantum Support Vector Machines (QSVM), Quantum Principal Component Analysis (QPCA), and Quantum Neural Networks (QNN), each of which leverages quantum mechanics to overcome the computational barriers faced by classical algorithms. The Quantum Approximate Optimization Algorithm (QAOA) is also highlighted for its ability to optimize machine learning models more effectively. While the theoretical benefits of Quantum Machine Learning (QML) are promising, the practical application of these techniques is currently limited by the constraints of existing quantum hardware. This research contributes to the emerging field of QML by examining its potential advantages and future implications in addressing complex data processing challenges.

An Open-Source ML-Based Full-Stack Optimization Framework for Machine Learning Accelerators

Parameterizable machine learning (ML) accelerators are the product of recent breakthroughs in ML. To fully enable their design space exploration (DSE), we propose a physical-design-driven, learning-based prediction framework for hardware-accelerated deep neural network (DNN) and non-DNN ML algorithms. It adopts a unified approach that combines power, performance, and area (PPA) analysis with frontend performance simulation, thereby achieving a realistic estimation of both backend PPA and system metrics such as runtime and energy. In addition, our framework includes a fully automated DSE technique, which optimizes backend and system metrics through an automated search of architectural and backend parameters. Experimental studies show that our approach consistently predicts backend PPA and system metrics with an average 7% or less prediction error for the ASIC implementation of two deep learning accelerator platforms, VTA and VeriGOOD-ML, in both a commercial 12 nm process and a research-oriented 45 nm process.

Factored Systolic Arrays Based on Radix-8 Multiplication for Machine Learning Acceleration

Factored Systolic Arrays Based on Radix-8 Multiplication for Machine Learning Acceleration

Survey of Security Issues in Memristor-Based Machine Learning Accelerators for RF Analysis

Survey of Security Issues in Memristor-Based Machine Learning Accelerators for RF Analysis

Brain-machine interactive neuromodulation research tool with edge AI computing

Closed-loop neuromodulation with intelligence methods has shown great potentials in providing novel neuro-technology for treating neurological and psychiatric diseases. Development of brain-machine interactive neuromodulation strategies could lead to breakthroughs in precision and personalized electronic medicine. The neuromodulation research tool integrating artificial intelligent computing and performing neural sensing and stimulation in real-time could accelerate the development of closed-loop neuromodulation strategies and translational research into clinical application. In this study, we developed a brain-machine interactive neuromodulation research tool (BMINT), which has capabilities of neurophysiological signals sensing, computing with mainstream machine learning algorithms and delivering electrical stimulation pulse by pulse in real-time. The BMINT research tool achieved system time delay under 3 ms, and computing capabilities in feasible computation cost, efficient deployment of machine learning algorithms and acceleration process. Intelligent computing framework embedded in the BMINT enable real-time closed-loop neuromodulation developed with mainstream AI ecosystem resources. The BMINT could provide timely contribution to accelerate the translational research of intelligent neuromodulation by integrating neural sensing, edge AI computing and stimulation with AI ecosystems.

Bridging the gap between machine learning and particle accelerator physics with high-speed, differentiable simulations

Machine learning has emerged as a powerful solution to the modern challenges in accelerator physics. However, the limited availability of beam time, the computational cost of simulations, and the high dimensionality of optimization problems pose significant challenges in generating the required data for training state-of-the-art machine learning models. In this work, we introduce heetah, a yorch-based high-speed differentiable linear beam dynamics code. heetah enables the fast collection of large datasets by reducing computation times by multiple orders of magnitude and facilitates efficient gradient-based optimization for accelerator tuning and system identification. This positions heetah as a user-friendly, readily extensible tool that integrates seamlessly with widely adopted machine learning tools. We showcase the utility of heetah through five examples, including reinforcement learning training, gradient-based beamline tuning, gradient-based system identification, physics-informed Bayesian optimization priors, and modular neural network surrogate modeling of space charge effects. The use of such a high-speed differentiable simulation code will simplify the development of machine learning-based methods for particle accelerators and fast-track their integration into everyday operations of accelerator facilities. Published by the American Physical Society 2024

Cerberus: Triple Mode Acceleration of Sparse Matrix and Vector Multiplication

The multiplication of sparse matrix and vector (SpMV) is one of the most widely used kernels in high-performance computing as well as machine learning acceleration for sparse neural networks. The design space of SpMV accelerators has two axes: algorithm and matrix representation. There have been two widely used algorithms and data representations. Two algorithms, scalar multiplication and dot product, can be combined with two sparse data representations, compressed sparse and bitmap formats for the matrix and vector. Although the prior accelerators adopted one of the possible designs, it is yet to be investigated which design is the best one across different hardware resources and workload characteristics. This paper first investigates the impact of design choices with respect to the algorithm and data representation. Our evaluation shows that no single design always outperforms the others across different workloads, but the two best designs (i.e., compressed sparse format and bitmap format with dot product) have complementary performance with trade-offs incurred by the matrix characteristics. Based on the analysis, this study proposes Cerberus, a triple-mode accelerator supporting two sparse operation modes in addition to the base dense mode. To allow such multi-mode operation, it proposes a prediction model based on matrix characteristics under a given hardware configuration, which statically selects the best mode for a given sparse matrix with its dimension and density information. Our experimental results show that Cerberus provides 12.1× performance improvements from a dense-only accelerator, and 1.5× improvements from a fixed best SpMV design.

Ferroelectric capacitors and field-effect transistors as in-memory computing elements for machine learning workloads

This study discusses the feasibility of Ferroelectric Capacitors (FeCaps) and Ferroelectric Field-Effect Transistors (FeFETs) as In-Memory Computing (IMC) elements to accelerate machine learning (ML) workloads. We conducted an exploration of device fabrication and proposed system-algorithm co-design to boost performance. A novel FeCap device, incorporating an interfacial layer (IL) and Hf0.5Zr0.5O2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ ext {Hf}_{0.5}\ ext {Zr}_{0.5}\ ext {O}_2$$\\end{document} (HZO), ensures a reduction in operating voltage and enhances HZO scaling while being compatible with CMOS circuits. The IL also enriches ferroelectricity and retention properties. When integrated into crossbar arrays, FeCaps and FeFETs demonstrate their effectiveness as IMC components, eliminating sneak paths and enabling selector-less operation, leading to notable improvements in energy efficiency and area utilization. However, it is worth noting that limited capacitance ratios in FeCaps introduced errors in multiply-and-accumulate (MAC) computations. The proposed co-design approach helps in mitigating these errors and achieves high accuracy in classifying the CIFAR-10 dataset, elevating it from a baseline of 10% to 81.7%. FeFETs in crossbars, with a higher on-off ratio, outperform FeCaps, and our proposed charge-based sensing scheme achieved at least an order of magnitude reduction in power consumption, compared to prevalent current-based methods.

M3ICRO: Machine learning-enabled compact photonic tensor core based on programmable multi-operand multimode interference

Photonic computing shows promise for transformative advancements in machine learning (ML) acceleration, offering ultrafast speed, massive parallelism, and high energy efficiency. However, current photonic tensor core (PTC) designs based on standard optical components hinder scalability and compute density due to their large spatial footprint. To address this, we propose an ultracompact PTC using customized programmable multi-operand multimode interference (MOMMI) devices, named M3ICRO. The programmable MOMMI leverages the intrinsic light propagation principle, providing a single-device programmable matrix unit beyond the conventional computing paradigm of one multiply-accumulate operation per device. To overcome the optimization difficulty of customized devices that often requires time-consuming simulation, we apply ML for optics to predict the device behavior and enable differentiable optimization flow. We thoroughly investigate the reconfigurability and matrix expressivity of our customized PTC and introduce a novel block unfolding method to fully exploit the computing capabilities of a complex-valued PTC for near-universal real-valued linear transformations. Extensive evaluations demonstrate that M3ICRO achieves a 3.5–8.9× smaller footprint, 1.6–4.4× higher speed, 9.9–38.5× higher compute density, 3.7–12× higher system throughput, and superior noise robustness compared to state-of-the-art coherent PTC designs. It also outperforms electronic digital A100 graphics processing unit by 34.8–403× higher throughput while maintaining close-to-digital task accuracy across various ML benchmarks.

Privacy‐preserving task offloading in mobile edge computing: A deep reinforcement learning approach

AbstractAs machine learning (ML) technologies continue to evolve, there is an increasing demand for data. Mobile crowd sensing (MCS) can motivate more users in the data collection process through reasonable compensation, which can enrich the data scale and coverage. However, nowadays, users are increasingly concerned about their privacy and are unwilling to easily share their personal data. Therefore, protecting privacy has become a crucial issue. In ML, federated learning (FL) is a widely known privacy‐preserving technique where the model training process is performed locally by the data owner, which can protect privacy to a large extent. However, as the model size grows, the weak computing power and battery life of user devices are not sufficient to support training a large number of models locally. With mobile edge computing (MEC), user can offload some of the model training tasks to the edge server for collaborative computation, allowing the edge server to participate in the model training process to improve training efficiency. However, edge servers are not fully trusted, and there is still a risk of privacy leakage if data is directly uploaded to the edge server. To address this issue, we design a local differential privacy (LDP) based data privacy‐preserving algorithm and a deep reinforcement learning (DRL) based task offloading algorithm. We also propose a privacy‐preserving distributed ML framework for MEC and model the cloud‐edge‐mobile collaborative training process. These algorithms not only enable effective utilization of edge computing to accelerate machine learning model training but also significantly enhance user privacy and save device battery power. We have conducted experiments to verify the effectiveness of the framework and algorithms.

Dread and the automation of education: From algorithmic anxiety to a new sensibility

Accelerating digitization, algorithmic computation, artificial intelligence, and machine learning, along with the increasing automation of work, communication, and everyday life, are central to critical studies of technology and political economy, as well as to public discourse concerning technology’s role in creating futures. Ongoing transformations in technological capacity have also been scrutinized for their impact on experience, emotion, and culture. Building on David Theo Goldberg’s assertion that “tracking capitalism” creates pervasive dread, this article explores how the digital automation of education, specifically, generates forms of algorithmic anxiety that impact teaching and learning, constraining pedagogical visions of alternative futures. Algorithmic anxiety in education contributes to dread’s proliferation. If dread is the “driving social sensibility” today, and algorithmic anxiety a pedagogical vector of its spread, then a less dreadful education should center an imaginative struggle for new sensibilities.

Beyond theory-driven discovery: introducing hot random search and datum-derived structures.

Data-driven methods have transformed the prospects of the computational chemical sciences, with machine-learned interatomic potentials (MLIPs) speeding up calculations by several orders of magnitude. I reflect on theory-driven, as opposed to data-driven, discovery based on ab initio random structure searching (AIRSS), and then introduce two new methods that exploit machine-learning acceleration. I show how long high-throughput anneals, between direct structural relaxation, enabled by ephemeral data-derived potentials (EDDPs), can be incorporated into AIRSS to bias the sampling of challenging systems towards low-energy configurations. Hot AIRSS (hot-AIRSS) preserves the parallel advantage of random search, while allowing much more complex systems to be tackled. This is demonstrated through searches for complex boron structures in large unit cells. I then show how low-energy carbon structures can be directly generated from a single, experimentally determined, diamond structure. An extension to the generation of random sensible structures, candidates are stochastically generated and then optimised to minimise the difference between the EDDP environment vector and that of the reference diamond structure. The distance-based cost function is captured in an actively learned EDDP. Graphite, small nanotubes and caged, fullerene-like, structures emerge from searches using this potential, along with a rich variety of tetrahedral framework structures. Using the same approach, the pyrope, Mg3Al2(SiO4)3, garnet structure is recovered from a low-energy AIRSS structure generated in a smaller unit cell with a different chemical composition. The relationship of this approach to modern diffusion-model-based generative methods is discussed.

Energy efficiency in edge TPU vs. embedded GPU for computer-aided medical imaging segmentation and classification

In this work, we evaluate the energy usage of fully embedded medical diagnosis aids based on both segmentation and classification of medical images implemented on Edge TPU and embedded GPU processors. We use glaucoma diagnosis based on color fundus images as an example to show the possibility of performing segmentation and classification in real time on embedded boards and to highlight the different energy requirements of the studied implementations.Several other works develop the use of segmentation and feature extraction techniques to detect glaucoma, among many other pathologies, with deep neural networks. Memory limitations and low processing capabilities of embedded accelerated systems (EAS) limit their use for deep network-based system training. However, including specific acceleration hardware, such as NVIDIA's Maxwell GPU or Google's Edge TPU, enables them to perform inferences using complex pre-trained networks in very reasonable times.In this study, we evaluate the timing and energy performance of two EAS equipped with Machine Learning (ML) accelerators executing an example diagnostic tool developed in a previous work. For optic disc (OD) and cup (OC) segmentation, the obtained prediction times per image are under 29 and 43 ms using Edge TPUs and Maxwell GPUs, respectively. Prediction times for the classification subsystem are lower than 10 and 14 ms for Edge TPUs and Maxwell GPUs, respectively. Regarding energy usage, in approximate terms, for OD segmentation Edge TPUs and Maxwell GPUs use 38 and 190 mJ per image, respectively. For fundus classification, Edge TPUs and Maxwell GPUs use 45 and 70 mJ, respectively.

Accelerating Machine Learning Inference with GPUs in ProtoDUNE Data Processing

We study the performance of a cloud-based GPU-accelerated inference server to speed up event reconstruction in neutrino data batch jobs. Using detector data from the ProtoDUNE experiment and employing the standard DUNE grid job submission tools, we attempt to reprocess the data by running several thousand concurrent grid jobs, a rate we expect to be typical of current and future neutrino physics experiments. We process most of the dataset with the GPU version of our processing algorithm and the remainder with the CPU version for timing comparisons. We find that a 100-GPU cloud-based server is able to easily meet the processing demand, and that using the GPU version of the event processing algorithm is two times faster than processing these data with the CPU version when comparing to the newest CPUs in our sample. The amount of data transferred to the inference server during the GPU runs can overwhelm even the highest-bandwidth network switches, however, unless care is taken to observe network facility limits or otherwise distribute the jobs to multiple sites. We discuss the lessons learned from this processing campaign and several avenues for future improvements.

Instruction-Level Power Side-Channel Leakage Evaluation of Soft-Core CPUs on Shared FPGAs

Side-channel disassembly attacks recover CPU instructions from power or electromagnetic side-channel traces measured during code execution. These attacks typically rely on physical access, proximity to the victim device, and high sampling rate measuring instruments. In this work, however, we analyze the CPU instruction-level power side-channel leakage in an environment that lacks physical access or expensive measuring equipment. We show that instruction leakage is present even in a multitenant FPGA scenario, where the victim uses a soft-core CPU, and the adversary deploys on-chip voltage-fluctuation sensors. Unlike previous remote power side-channel attacks, which either require a considerable number of victim traces or attack large victim circuits such as machine learning accelerators, we take an evaluator’s point of view and provide an analysis of the instruction-level power side-channel leakage of a small open-source RISC-V soft processor core. To investigate whether the power side-channel traces leak secrets, we profile the victim device and implement various instruction opcode classifiers based on both classical machine learning algorithms used in disassembly attacks, and novel, deep learning approaches. We explore how parameters such as placement, trace averaging, profiling templates, and different FPGA families (including a cloud-scale FPGA) impact the classification accuracy. Despite the limited leakage of the soft-core CPU victim and a reduced accuracy and sampling rate of on-chip sensors, we show that in a worst-case scenario for the evaluator, i.e., an attacker breaching physical separation, we can identify the opcode of executed instructions with an average accuracy as high as 86.46%. Our analysis shows that determining the executed instruction type is not a classification bottleneck, while leakages between instructions of the same type can be challenging for deep learning models to distinguish. We also show that the instruction-level leakage is significantly reduced in a cloud-scale FPGA scenario with higher soft-core CPU frequencies. Nevertheless, our results show that even small circuits, such as soft-core CPUs, leak potentially exploitable information through on-chip power side channels, and users should deploy mitigation techniques against disassembly attacks to protect their proprietary code and data.

SAMBA: Sparsity Aware In-Memory Computing Based Machine Learning Accelerator

Machine Learning (ML) inference is typically dominated by highly data-intensive Matrix Vector Multiplication (MVM) computations that may be constrained by the memory bottleneck due to massive data movement between processor and memory units. Although analog in-memory computing (IMC) ML accelerators have been proposed to execute MVM with high efficiency, the latency and energy of such analog computing systems can be dominated by the large latency and energy costs from analog-to-digital converters (ADC). Leveraging sparsity in ML workloads, reconfigurable ADCs can improve the MVM energy and latency by reducing the required ADC bit precision. However, such improvement in latency can be hindered by non-uniform sparsity of the weight matrices mapped into hardware. Moreover, data movement between MVM processing cores may become another factor that delays the overall system-level performance. To address these issues, we propose SAMBA, Sparsity Aware IMC Based Machine Learning Accelerator. First, we propose load balancing at the mapping of weight matrices into physical crossbars. The goal of load balancing is to accommodate reconfigurable ADCs and address the possible delay of MVM processing caused by non-uniform sparsity in the weight matrices. Second, we propose optimizations in arranging and scheduling the tiled MVM hardware to minimize the overhead of data movement across multiple processing cores. Our evaluations show that the proposed load balancing technique can achieve performance improvement by eliminating the non-uniformity in the sparsity of mapped matrices. Moreover, we demonstrate that data movement optimizations can further improve both performance and energy efficiency regardless of sparsity condition. With the combination of load balancing and data movement optimization in conjunction with reconfigurable ADCs, our proposed approach provides up to 2.38x speed-up and 1.54x energy efficiency over state-of-art analog IMC based ML accelerators for ImageNet datasets on Resnet-50 architecture.

FPGA-Based Machine Learning: Platforms, Applications, Design Considerations, Challenges, and Future Directions

Field-Programmable Gate Arrays (FPGAs) have emerged as a promising platform for accelerating machine learning tasks due to their high parallelism, low latency, and hardware customization ability. In this paper, the authors provide an overview of popular FPGA platforms for machine learning and compare the tradeoffs among FPGAs, GPUs, and CPUs for machine learning. The authors also present specific applications of machine learning based on FPGAs, including those in autonomous driving and healthcare. Additionally, the paper explores FPGA design considerations, such as architecture, resource utilization, and power consumption. Nonetheless, obstacles persist in the realm of FPGA-based machine learning that require attention. Identifying the ideal balance between adaptability and performance, considering factors such as space, energy usage, and latency, is still challenging. As the capabilities of FPGAs expand, there is a significant need for devices that have a smaller footprint, reduced power consumption, and minimized delays. The paper emphasizes the necessity of ongoing research in the field of FPGA-based machine learning to address these issues and continue enhancing the performance and effectiveness of machine learning systems.

Bottom-Up and Top-Down Approaches for the Design of Neuromorphic Processing Systems: Tradeoffs and Synergies Between Natural and Artificial Intelligence

While Moore's law has driven exponential computing power expectations, its nearing end calls for new avenues for improving the overall system performance. One of these avenues is the exploration of alternative brain-inspired computing architectures that aim at achieving the flexibility and computational efficiency of biological neural processing systems. Within this context, neuromorphic engineering represents a paradigm shift in computing based on the implementation of spiking neural network architectures in which processing and memory are tightly co-located. In this paper, we provide a comprehensive overview of the field, highlighting the different levels of granularity at which this paradigm shift is realized and comparing design approaches that focus on replicating natural intelligence (bottom-up) versus those that aim at solving practical artificial intelligence applications (top-down). First, we present the analog, mixed-signal and digital circuit design styles, identifying the boundary between processing and memory through time multiplexing, in-memory computation, and novel devices. Then, we highlight the key tradeoffs for each of the bottom-up and top-down design approaches, survey their silicon implementations, and carry out detailed comparative analyses to extract design guidelines. Finally, we identify necessary synergies and missing elements required to achieve a competitive advantage for neuromorphic systems over conventional machine-learning accelerators in edge computing applications, and outline the key ingredients for a framework toward neuromorphic intelligence.