Custom Accelerators Research Articles

Optimizing Neural Network Inference in Edge Robotics by Harnessing FPGA Hardware Acceleration

This research explores advancements in neural network inference for edge robotics through the implementation of hardware acceleration using Field-Programmable Gate Arrays (FPGA). Leveraging FPGA's versatility in custom hardware design, the research optimizes performance and energy efficiency by utilizing features such as Digital Signal Processors (DSP), Multiply-Accumulate (MAC) arrays, parallel processing units, Convolutional Neural Networks (CNN), and custom accelerators. The computational power of FPGA-based architectures is harnessed to significantly enhance the speed and efficiency of neural network inference tasks in edge robotics applications. Key aspects include the utilization of DSP blocks for efficient signal processing, MAC arrays for high-speed matrix operations crucial in neural network computations, and parallel processing units for concurrent execution of multiple tasks, improving overall system throughput. The integration of CNN directly onto FPGA hardware allows for minimal-latency inference, making it ideal for real-time applications in edge robotics scenarios. Custom accelerators tailored to specific neural network operations further optimize performance and resource utilization. The research involves the meticulous design and synthesis of hardware modules tailored to the target FPGA platform. Through rigorous optimization and parallelization techniques, the goal is to maximize resource utilization while minimizing latency and power consumption. The experimental findings showcase remarkable results, including a substantial increase in throughput, with values reaching up to 150 images/sec, and impressive accuracy rates exceeding 95% in classification tasks.

DANSEN: Database Acceleration on Native Computational Storage by Exploiting NDP

This paper introduces DANSEN , the hardware accelerator component for neoDBMS, a full-stack computational storage system designed to manage on-device execution of database queries/transactions as a Near-Data Processing (NDP)-operation. The proposed system enables Database Management Systems (DBMS) to offload NDP-operations to the storage while maintaining control over data through a native storage interface . DANSEN provides an NDP-engine that enables DBMS to perform both low-level database tasks, such as performing database administration, as well as high-level tasks like executing SQL, on the smart storage device while observing the DBMS concurrency control. Furthermore, DANSEN enables the incorporation of custom accelerators as an NDP-operation, e.g., to perform hardware-accelerated ML inference directly on the stored data. We built the DANSEN storage prototype and interface on an Ultrascale+HBM FPGA and fully integrated it with PostgreSQL 12. Experimental results demonstrate that the proposed NDP approach outperforms software-only PostgreSQL using a fast off-the-shelf NVMe drive, and significantly improves the end-to-end execution time of an aggregation operation (similar to Q6 from CH-benCHmark, 150 million records) by ≈ 10.6 ×. The versatility of the proposed approach is also validated by integrating a compute-intensive data analytics application with multi-row results, outperforming PostgreSQL by ≈ 1.5 ×.

Security Enabled New Term Weight Measure Technique with Data Driven for Next Generation Mobile Computing Networks

In the field of ASIC and FPGA, Machine Learning (ML) techniques play a major role and become predominant for accurate results for different applications like big data analysis and automotive electronics, and driverless vehicles which are required speed and power savings. Due to increasing the demand for higher accuracy, low power, low area consumption, and higher throughput for the complexity of the designs in the latest technology, the proposed system is fulfilling these demands in ASIC and FPGA domains, reconfigurable hardware architecture has been proposed it consists of an ML-based Support Vector Machine (SVM), high-speed AHB protocol and Floating point (FP) operations and also the system has the flexibility to communicate with I2C and I2S protocols. In order to increase throughput with minimal latency, the proposed architecture with AHB protocol and AHB to APB bridge is incorporated between the fabric dynamically reconfigurable multi-processor (FDPM) and peripherals along with security algorithms using SHA-256bits and AES. In order to perform ML-based applications, the proposed system is incorporated double-precision floating point (DPFP) arithmetic operations. The overall proposed architecture is developed in Verilog HDL and quality checking using the LINT tool and Clock Domain Crossing (CDC) using Spyglass tool and synthesized using DC compiler for ASIC and Vivado Design Suite 2018.1 for FPGA implementation and verification. The entire design is interfaced with the Zynq processor and SDK tool to verify data transfer between hardware and software. The obtained results show the generated custom accelerator is able to compute any complex ML classifiers for a larger amount of data. The obtained results are compared with existing state-of-art results and found that 18 % improvement in throughput, a 21 % improvement in power consumption savings, and a 34 % reduction in latency.

Hardware Implementations of a Deep Learning Approach to Optimal Configuration of Reconfigurable Intelligence Surfaces

Reconfigurable intelligent surfaces (RIS) offer the potential to customize the radio propagation environment for wireless networks, and will be a key element for 6G communications. However, due to the unique constraints in these systems, the optimization problems associated to RIS configuration are challenging to solve. This paper illustrates a new approach to the RIS configuration problem, based on the use of artificial intelligence (AI) and deep learning (DL) algorithms. Concretely, a custom convolutional neural network (CNN) intended for edge computing is presented, and implementations on different representative edge devices are compared, including the use of commercial AI-oriented devices and a field-programmable gate array (FPGA) platform. This FPGA option provides the best performance, with ×20 performance increase over the closest FP32, GPU-accelerated option, and almost ×3 performance advantage when compared with the INT8-quantized, TPU-accelerated implementation. More noticeably, this is achieved even when high-level synthesis (HLS) tools are used and no custom accelerators are developed. At the same time, the inherent reconfigurability of FPGAs opens a new field for their use as enabler hardware in RIS applications.

Diagnosis of Malicious Bitstreams in Cloud Computing FPGAs

Multi-tenant FPGAs are increasingly being used in cloud computing technologies. Users are able to access the FPGA fabric remotely to implement custom accelerators in the cloud. However, the sharing of FPGA resources by untrusted third-parties can lead to serious security threats. Attackers can configure a portion of the FPGA with a malicious bitstream. Such malicious use of the FPGA fabric may lead to severe voltage fluctuations and denial-of-service. In this work, we consider FPGAs that support time-based multi-tenancy i.e., a single user has access to the FPGA at a time. We propose a convolutional neural network (CNN)-based approach to detect malicious RO-like circuits that are configured on an FPGA by learning features from the data-series representation of the bitstreams of malicious circuits. We use the classification accuracy, true-positive rate, and false-positive rate metrics to quantify the effectiveness of CNN-based classification of malicious bitstreams. Our threat model includes a variety of power-wasting circuits that are used to configure FPGAs in the cloud. We propose a two-stage malicious bitstream detection framework for classification and diagnosis of the type of malicious circuit implemented by a particular bitstream. We further propose a novel window-merging technique to improve model performance in the second stage of the detection framework. Experimental results on Xilinx FPGAs demonstrate the effectiveness of the proposed method.

Acceleration of Control Intensive Applications on Coarse-Grained Reconfigurable Arrays for Embedded Systems

Embedded systems confront two opposite goals: low-power operation and high performance. The current trend to reach these goals is toward heterogeneous platforms, including multi-core architectures with heterogeneous cores and hardware accelerators. The latter can be divided into custom accelerators (e.g., ASICs) and programmable domain-specific cores (e.g., DSIPs). VWR2A [1] is a programmable architecture that integrates high computational density and low power memory structures. The flexibility of VWR2A allows a large portion of applications to be covered, resulting in better performance and energy efficiency than ASICs and general-purpose processors. However, while this has been well studied for data-intensive kernels, this is not the case for control-intensive kernels —code with complex if-else and nested loop structures. Traditionally, control-intensive code is left to be executed by the host processor. This situation unnecessarily restricts the potential impact of energy-efficient acceleration, especially at the application level. In this paper, we evaluate the performance and energy consumption of VWR2A for control-intensive code and compare it with an ARM Cortex-M4 processor and a RISC-V Ibex processor. The performance and energy consumption are evaluated at the kernel and application levels. Our results confirm that VWR2A is faster and more energy-efficient than the two considered general-purpose processors also for control-intensive code.

Deep Learning Inferencing with High-performance Hardware Accelerators

As computer architectures continue to integrate application-specific hardware, it is critical to understand the relative performance of devices for maximum app acceleration. The goal of benchmarking suites, such as MLPerf for analyzing machine learning (ML) hardware performance, is to standardize a fair comparison of different hardware architectures. However, there are many apps that are not well represented by these standards that require different workloads, such as ML models and datasets, to achieve similar goals. Additionally, many apps, like real-time video processing, are focused on latency of computations rather than strictly on throughput. This research analyzes multiple compute architectures that feature ML-specific hardware on a case study of handwritten Chinese character recognition. Specifically, AlexNet and a custom version of GoogLeNet are benchmarked in terms of their streaming latency and maximum throughput for optical character recognition. Considering that these models are composed of fundamental neural network operations yet architecturally different from each other, these models can stress devices in different yet insightful ways that generalizations of the performance of other models can be drawn from. Many devices featuring ML-specific hardware and optimizations are analyzed including Intel and AMD CPUs, Xilinx and Intel FPGAs, NVIDIA GPUs, and Google TPUs. Overall, ML-oriented hardware added to the Intel Xeon CPUs helps to boost throughput by 3.7× and to reduce latency by up to 34.7×, which makes the latency of Intel Xeon CPUs competitive on more parallel models. The TPU devices were limited in terms of throughput due to large data transfer times and not competitive in terms of latency. The FPGA frameworks showcase the lowest latency on the Xilinx Alveo U200 FPGA achieving 0.48 ms on AlexNet using Mipsology Zebra and 0.39 ms on GoogLeNet using Vitis-AI. Through their custom acceleration datapaths coupled with high-performance SRAM, the FPGAs are able to keep critical model data closer to processing elements for lower latency. The massively parallel and high-memory GPU devices with Tensor Core accelerators achieve the best throughput. The NVIDIA Tesla A100 GPU showcases the highest throughput at 42,513 and 52,484 images/second for AlexNet and GoogLeNet, respectively. 1

A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm

The energy efficiency of deep neural network (DNN) inference can be improved with custom accelerators. DNN inference accelerators often employ specialized hardware techniques to improve energy efficiency, but many of these techniques result in catastrophic accuracy loss on transformer-based DNNs, which have become ubiquitous for natural language processing (NLP) tasks. This article presents a DNN accelerator designed for efficient execution of transformers. The proposed accelerator implements per-vector scaled quantization (VSQ), which employs an independent scale factor for each 64-element vector to enable the use of 4-bit arithmetic with little accuracy loss and low energy overhead. Using a multilevel dataflow to maximize reuse, the 5-nm prototype achieves 95.6 tera-operations per second per Watt (TOPS/W) at 0.46 V on a 4-bit benchmarking layer with VSQ. At a nominal voltage of 0.67 V, the accelerator achieves 1734 inferences/s/W (38.7 TOPS/W) with only 0.7% accuracy loss on BERT-Base and 4714 inferences/s/W (38.6 TOPS/W) with 0.15% accuracy loss on ResNet-50 by using quantization-aware fine-tuning to recover accuracy, demonstrating a practical accelerator design for energy-efficient DNN inference.

Combining Image Space and q-Space PDEs for Lossless Compression of Diffusion MR Images

Diffusion MRI is a modern neuroimaging modality with a unique ability to acquire microstructural information by measuring water self-diffusion at the voxel level. However, it generates huge amounts of data, resulting from a large number of repeated 3D scans. Each volume samples a location in q-space, indicating the direction and strength of a diffusion sensitizing gradient during the measurement. This captures detailed information about the self-diffusion and the tissue microstructure that restricts it. Lossless compression with GZIP is widely used to reduce the memory requirements. We introduce a novel lossless codec for diffusion MRI data. It reduces file sizes by more than 30% compared to GZIP and also beats lossless codecs from the JPEG family. Our codec builds on recent work on lossless PDE-based compression of 3D medical images, but additionally exploits smoothness in q-space. We demonstrate that, compared to using only image space PDEs, q-space PDEs further improve compression rates. Moreover, implementing them with finite element methods and a custom acceleration significantly reduces computational expense. Finally, we show that our codec clearly benefits from integrating subject motion correction and slightly from optimizing the order in which the 3D volumes are coded.

A Framework for Design, Verification, and Management of SoC Access Control Systems

System-on-chip (SoC) architectures are a heterogeneous mix of microprocessors, custom accelerators, memories, interfaces, peripherals, and other resources. These resources communicate using complex on-chip interconnect networks that attempt to quickly and efficiently arbitrate memory transactions whose behaviors can vary drastically depending on the current mode of operation and system operating state. Security- and safety-critical applications require access control policies that define how these resources interact to ensure that malicious and unsafe behaviors do not occur. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Aker</small> is a design and verification framework for on-chip access control. The core of <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Aker</small> is the access control wrapper (ACW)–a high-performance yet efficient hardware module that dynamically arbitrates on-chip communications. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Aker</small> distributes ACWs across the SoC and programs them to perform local access control. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Aker</small> provides a firmware generation tool and a property-driven security verification methodology to ensure that the ACWs are properly integrated and configured. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Aker</small> security verification confirms that the ACW behaves properly at IP level. It verifies the hardware root of trust firmware configures the ACW correctly. And it evaluates system-level security threats due to interactions between shared resources. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Aker</small> is experimentally validated on a Xilinx UltraScale+ programmable SoC. Additionally, an <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Aker</small> access control system is integrated into the OpenPULP multicore archtiecture that uses OpenTitan hardware root-of-trust for firmware configuration.

Accelerating Large-Scale Graph-Based Nearest Neighbor Search on a Computational Storage Platform

K-nearest neighbor search is one of the fundamental tasks in various applications and the hierarchical navigable small world (HNSW) has recently drawn attention in large-scale cloud services, as it easily scales up the database while offering fast search. On the other hand, a computational storage device (CSD) that combines programmable logic and storage modules on a single board becomes popular to address the data bandwidth bottleneck of modern computing systems. In this paper, we propose a computational storage platform that can accelerate a large-scale graph-based nearest neighbor search algorithm based on SmartSSD CSD. To this end, we modify the algorithm more amenable on the hardware and implement two types of accelerators using HLS- and RTL-based methodology with various optimization methods. In addition, we scale up the proposed platform to have 4 SmartSSDs and apply graph parallelism to boost the system performance further. As a result, the proposed computational storage platform achieves 75.59 query per second throughput for the SIFT1B dataset at 258.66W power dissipation, which is 12.83x and 17.91x faster and 10.43x and 24.33x more energy efficient than the conventional CPU-based and GPU-based server platform, respectively. With multi-terabyte storage and custom acceleration capability, we believe that the proposed computational storage platform is a promising solution for cost-sensitive cloud datacenters.

A Unified Programmable Edge Matrix Processor for Deep Neural Networks and Matrix Algebra

Matrix Algebra and Deep Neural Networks represent foundational classes of computational algorithms across multiple emerging applications like Augmented Reality or Virtual Reality, autonomous navigation (cars, drones, robots), data science, and various artificial intelligence-driven solutions. An accelerator-based architecture can provide performance and energy efficiency supporting fixed functions through customized data paths. However, constrained Edge systems requiring multiple applications and diverse matrix operations to be efficiently supported, cannot afford numerous custom accelerators. In this article, we present MxCore, a unified architecture that comprises tightly coupled vector and programmable cores sharing data through highly optimized interconnects along with a configurable hardware scheduler managing the co-execution. We submit MxCore as the generalized approach to facilitate the flexible acceleration of multiple Matrix Algebra and Deep-learning applications across a range of sparsity levels. Unified compute resources improve overall resource utilization and performance per unit area. Aggressive and novel microarchitecture techniques along with block-level sparsity support optimize compute and data-reuse to minimize bandwidth and power requirements enabling ultra-low latency applications for low-power and cost-sensitive Edge deployments. MxCore requires a small silicon footprint of 0.2068 mm 2 , in a modern 7-nm process at 1 GHz and achieves (0.15 FP32 and 0.62 INT8) TMAC/mm 2 , dissipating only 11.66 μW of leakage power. At iso-technology and iso-frequency, MxCore provides an energy efficiency of 651.4×, 159.9×, 104.8×, and 124.2× as compared to the 128-core Nvidia’s Maxwell GPU for dense General Matrix Multiply, sparse Deep Neural Network, Cholesky decomposition, and triangular matrix solve respectively.

NTT Architecture for a Linux-Ready RISC-V Fully-Homomorphic Encryption Accelerator

This paper proposes two architectures for the acceleration of Number Theoretic Transforms (NTTs) using a novel Montgomery-based butterfly. We first design a custom NTT hardware accelerator for Field-Programmable Gate Arrays (FPGAs). The butterfly architecture is expanded to a Modular Arithmetic Logic Unit (MALU) and for greater reuse and easier programmability a six-stage pipeline Linux-ready RISC-V core is extended with custom instructions. The performance of the proposed architectures is assessed on a Xilinx Ultrascale+ FPGA and with an Application-Specific Integrated Circuit (ASIC) on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$28{n}\text{m}$ </tex-math></inline-formula> CMOS technology. In FPGA, the results for custom acceleration show reductions of 30%, 90% and 42% in the number of Lookup tables (LUTs) and registers, Block RAMs (BRAMs) and Digital Signal Processors (DSPs), while providing a speedup of 1.9 times, in comparison with the state of the art. The ASIC results show that at 1 GHz the proposed architecture is in average 45% and 52% less area and power hungry, respectively, compared to the state of the art. Furthermore, the proposed MALU, operating as an additional execution unit, increases the overall area of the extended RISC-V core by only 10%, without significant changes in the frequency of operation.

Embedded Object Detection with Custom LittleNet, FINN and Vitis AI DCNN Accelerators

Object detection is an essential component of many systems used, for example, in advanced driver assistance systems (ADAS) or advanced video surveillance systems (AVSS). Currently, the highest detection accuracy is achieved by solutions using deep convolutional neural networks (DCNN). Unfortunately, these come at the cost of a high computational complexity; hence, the work on the widely understood acceleration of these algorithms is very important and timely. In this work, we compare three different DCNN hardware accelerator implementation methods: coarse-grained (a custom accelerator called LittleNet), fine-grained (FINN) and sequential (Vitis AI). We evaluate the approaches in terms of object detection accuracy, throughput and energy usage on the VOT and VTB datasets. We also present the limitations of each of the methods considered. We describe the whole process of DNNs implementation, including architecture design, training, quantisation and hardware implementation. We used two custom DNN architectures to obtain a higher accuracy, higher throughput and lower energy consumption. The first was implemented in SystemVerilog and the second with the FINN tool from AMD Xilinx. Next, both approaches were compared with the Vitis AI tool from AMD Xilinx. The final implementations were tested on the Avnet Ultra96-V2 development board with the Zynq UltraScale+ MPSoC ZCU3EG device. For two different DNNs architectures, we achieved a throughput of 196 fps for our custom accelerator and 111 fps for FINN. The same networks implemented with Vitis AI achieved 123.3 fps and 53.3 fps, respectively.

End-to-End Synthesis of Dynamically Controlled Machine Learning Accelerators

Edge systems are required to autonomously make real-time decisions based on large quantities of input data under strict power, performance, area, and other constraints. Meeting these constraints is only possible by specializing systems through hardware accelerators purposefully built for machine learning and data analysis algorithms. However, data science evolves at a quick pace, and manual design of custom accelerators has high non-recurrent engineering costs: general solutions are needed to automatically and rapidly transition from the formulation of a new algorithm to the deployment of a dedicated hardware implementation. Our solution is the SOftware Defined Architectures (SODA) Synthesizer, an end-to-end, multi-level, modular, extensible compiler toolchain providing a direct path from machine learning tools to hardware. The SODA Synthesizer frontend is based on the multilevel intermediate representation (MLIR) framework; it ingests pre-trained machine learning models, identifies kernels suited for acceleration, performs high-level optimizations, and prepares them for hardware synthesis. In the backend, SODA leverages state-of-the-art high-level synthesis techniques to generate highly efficient accelerators, targeting both field programmable devices (FPGAs) and application-specific circuits (ASICs). In this paper, we describe how the SODA Synthesizer can also assemble the generated accelerators (based on the finite state machine with datapath model) in a custom system driven by a distributed controller, building a coarse-grained dataflow architecture that does not require a host processor to orchestrate parallel execution of multiple accelerators. We show the effectiveness of our approach by automatically generating ASIC accelerators for layers of popular deep neural networks (DNNs). Our high-level optimizations result in up to 74x speedup on isolated accelerators for individual DNN layers, and our dynamically scheduled architecture yields an additional 3x performance improvement when combining accelerators to handle streaming inputs.

Scalable Phylogeny Reconstruction with Disaggregated Near-memory Processing

Disaggregated computer architectures eliminate resource fragmentation in next-generation datacenters by enabling virtual machines to employ resources such as CPUs, memory, and accelerators that are physically located on different servers. While this paves the way for highly compute- and/or memory-intensive applications to potentially deploy all CPUs and/or memory resources in a datacenter, it poses a major challenge to the efficient deployment of hardware accelerators: input/output data can reside on different servers than the ones hosting accelerator resources, thereby requiring time- and energy-consuming remote data transfers that diminish the gains of hardware acceleration. Targeting a disaggregated datacenter architecture similar to the IBM dReDBox disaggregated datacenter prototype, the present work explores the potential of deploying custom acceleration units adjacently to the disaggregated-memory controller on memory bricks (in dReDBox terminology), which is implemented on FPGA technology, to reduce data movement and improve performance and energy efficiency when reconstructing large phylogenies (evolutionary relationships among organisms). A fundamental computational kernel is the Phylogenetic Likelihood Function (PLF), which dominates the total execution time (up to 95%) of widely used maximum-likelihood methods. Numerous efforts to boost PLF performance over the years focused on accelerating computation; since the PLF is a data-intensive, memory-bound operation, performance remains limited by data movement, and memory disaggregation only exacerbates the problem. We describe two near-memory processing models, one that addresses the problem of workload distribution to memory bricks, which is particularly tailored toward larger genomes (e.g., plants and mammals), and one that reduces overall memory requirements through memory-side data interpolation transparently to the application, thereby allowing the phylogeny size to scale to a larger number of organisms without requiring additional memory.

Leakage Reuse for Energy Efficient Near-Memory Computing of Heterogeneous DNN Accelerators

The exploration of custom deep neural network (DNN) accelerators for highly energy constrained edge devices with on-device intelligence is gaining traction in the research community. Despite the superior throughput and performance of custom accelerators as compared to CPUs or GPUs, the energy efficiency and versatility of state-of-the-art DNN accelerators is constrained due to a) the storage and movement of a large volume of data and b) the limited scope of monolithic architectures, where the entire accelerator executes only a single model at any given time. In this paper, a multi-voltage domain heterogeneous DNN accelerator is proposed that executes multiple models simultaneously with different power-performance operating points. The proposed architecture concurrently implements near-memory computing and leakage reuse, where the leakage current of idle memory banks within each processing element is utilized to deliver current to the adjacently placed multiply-and-accumulate (MAC) units. The proposed architecture and circuit techniques are evaluated with SPICE simulation in a 65 nm CMOS technology. The simulation results indicate that the proposed heterogeneous architecture with leakage reuse improves the energy efficiency to 3.27 tera-operations per second per watt (TOPS/W) as compared to a conventional monolithic and single voltage domain architecture that exhibits an energy efficiency of 0.0458 TOPS/W. In addition, the proposed accelerator that applies the leakage reuse technique to only half of the memory elements storing the weights reduces the power consumption of the sub-arrays of processing elements by 100 mW as compared to an accelerator that does not apply leakage reuse.

Svelto: High-Level Synthesis of Multi-Threaded Accelerators for Graph Analytics

Graph analytics are an emerging class of irregular applications. Operating on very large datasets, they present unique behaviors, such as fine-grained, unpredictable memory accesses, and highly unbalanced task level parallelism, that make existing high-performance general-purpose processors or accelerators (e.g., GPUs) suboptimal. To address these issues, research and industry are developing a variety of custom accelerator designs for this application area, including solutions based on reconfigurable devices (Field Programmable Gate Arrays). These new approaches often employ High-Level Synthesis (HLS) to Speed up the development of the accelerators. In this paper, we propose a novel architecture template for the automatic generation of accelerators for graph analytics and irregular applications. The architecture template includes a dynamic task scheduling mechanism, a parallel array of accelerators that enables supporting task-level parallelism with context switching, and a related multi-channel memory interface that decouples communication from computation and provides support for fine-grained atomic memory operations. We discuss the integration of the architectural template in an HLS flow, presenting the necessary modifications to enable automatic generation of the custom architectures starting from OpenMP annotated code. We evaluate our approach first by synthesizing and exploring triangle counting, a common graph algorithm, and then by synthesizing custom designs for a set of graph database benchmark queries, representing series of graph pattern matching routines. We compare the synthesized accelerators with previous state-of-the-art methodologies for the synthesis of parallel architectures, showing that the proposed approach allows reducing resource usage by optimizing the number of accelerators replicas without any performance penalty.

NZESPA: A Near-3D-Memory Zero Skipping Parallel Accelerator for CNNs

Convolutional neural networks (CNNs) are one of the most popular machine learning tools for computer vision. The ubiquitous use in several applications with its high computation-cost has made it lucrative for optimization through accelerated architecture. State-of-the-art has either exploited the parallelism of CNNs, or eliminated computations through sparsity or used near-memory processing (NMP) to accelerate the CNNs. We introduce NMP-fully sparse architecture, which acquires all three capabilities. The proposed architecture is parallel and hence processes the independent CNN tasks concurrently. To exploit the sparsity, the proposed system employs a dataflow, namely, Near-3D-Memory Zero Skipping Parallel dataflow or nZESPA dataflow. This dataflow maintains the compressed-sparse encoding of data that skips all ineffectual zero-valued computations of CNNs. We design a custom accelerator which employs the nZESPA dataflow. The grids of nZESPA modules are integrated into the logic layer of the hybrid memory cube. This integration saves a significant amount of off-chip communications while implementing the concept of NMP. We compare the proposed architecture with three other architectures which either do not exploit sparsity (NMP-dense) or do not employ NMP (traditional-fully sparse) or do not include both (traditional-dense). The proposed system outperforms the baselines in terms of performance and energy consumption while executing CNN inference.

A 0.32–128 TOPS, Scalable Multi-Chip-Module-Based Deep Neural Network Inference Accelerator With Ground-Referenced Signaling in 16 nm

Custom accelerators improve the energy efficiency, area efficiency, and performance of deep neural network (DNN) inference. This article presents a scalable DNN accelerator consisting of 36 chips connected in a mesh network on a multi-chip-module (MCM) using ground-referenced signaling (GRS). While previous accelerators fabricated on a single monolithic chip are optimal for specific network sizes, the proposed architecture enables flexible scaling for efficient inference on a wide range of DNNs, from mobile to data center domains. Communication energy is minimized with large on-chip distributed weight storage and a hierarchical network-on-chip and network-on-package, and inference energy is minimized through extensive data reuse. The 16-nm prototype achieves 1.29-TOPS/mm2 area efficiency, 0.11 pJ/op (9.5 TOPS/W) energy efficiency, 4.01-TOPS peak performance for a one-chip system, and 127.8 peak TOPS and 1903 images/s ResNet-50 batch-1 inference for a 36-chip system.