Application-specific Accelerators Research Articles

Vector Extensions in COTS Processors to Increase Guaranteed Performance in Real-Time Systems

The need for increased application performance in high-integrity systems such as those in avionics is on the rise as software continues to implement more complex functionalities. The prevalent computing solution for future high-integrity embedded products is multi-processor systems-on-chip (MPSoC) processors. MPSoCs include central processing unit (CPU) multicores that enable improving performance via thread-level parallelism. MPSoCs also include generic accelerators (graphics processing units [GPUs]) and application-specific accelerators. However, the data processing approach (DPA) required to exploit each of these underlying parallel hardware blocks carries several open challenges to enable the safe deployment in high-integrity domains. The main challenges include the qualification of its associated runtime system and the difficulties in analyzing programs deploying the DPA with out-of-the-box timing analysis and code coverage tools. In this work, we perform a thorough analysis of vector extensions (VExts) in current commercial off-the-shelf (COTS) processors for high-integrity systems. We show that VExts prevent many of the challenges arising with parallel programming models and GPUs. Unlike other DPAs, VExts require no runtime support, prevent design race conditions that might arise with parallel programming models, and have minimum impact on the software ecosystem, enabling the use of existing code coverage and timing analysis tools. We develop vectorized versions of neural network kernels and show that the NVIDIA Xavier VExts provide a reasonable increase in guaranteed application performance of up to 2.7x. Our analysis contends that VExts are the DPA approach with arguably the fastest path for adoption in high-integrity systems.

Multi-Objective Surrogate-Model-Based Neural Architecture and Physical Design Co-Optimization of Energy Efficient Neural Network Hardware Accelerators

In this paper, we propose a methodology for co-optimizing application specific neural network (NN) accelerators for accuracy and energy expenditure per inference. The architecture of the NN is co-optimized with the concrete ASIC implementation of the accelerator to provide reliable estimates of the energy efficiency. While not constrained to a specific application or NN accelerator architecture, the method is demonstrated on an application specific NN accelerator for the detection of atrial fibrillation in human electrocardiograms that is implemented in 22FDX/FDSOI technology. The NN accelerator is highly parameterizable, i.e., it can map NNs with different architectural properties to a synthesizeable register transfer level representation. The parameter space is further expanded by the parameters of the physical implementation (e.g. memories, clocking, voltage domains). Since the evaluation of accuracy and energy efficiency for a specific parameter combination is computationally expensive, different hyperparameter optimization methods are used and evaluated, including Bayesian Optimization, which tries to find the optimal neural network architecture and physical implementation parameters with a minimum number of training, simulation and evaluation steps.

A High-Throughput, Resource-Efficient Implementation of the RoCEv2 Remote DMA Protocol and its Application

The use of application-specific accelerators in data centers has been the state of the art for at least a decade, starting with the availability of General Purpose GPUs achieving higher performance either overall or per watt. In most cases, these accelerators are coupled via PCIe interfaces to the corresponding hosts, which leads to disadvantages in interoperability, scalability and power consumption. As a viable alternative to PCIe-attached FPGA accelerators this paper proposes standalone FPGAs as Network-attached Accelerators (NAAs) . To enable reliable communication for decoupled FPGAs we present an RDMA over Converged Ethernet v2 (RoCEv2) communication stack for high-speed and low-latency data transfer integrated into a hardware framework. For NAAs to be used instead of PCIe coupled FPGAs the framework must provide similar throughput and latency with low resource usage. We show that our RoCEv2 stack is capable of achieving 100 Gb/s throughput with latencies of less than 4μs while using about 10% of the available resources on a mid-range FPGA. To evaluate the energy efficiency of our NAA architecture, we built a demonstrator with 8 NAAs for machine learning based image classification. Based on our measurements, network-attached FPGAs are a great alternative to the more energy-demanding PCIe-attached FPGA accelerators.

Verification of Chisel Hardware Designs with ChiselVerify

With the current ever-increasing demand for performance, hardware developers find themselves turning ever-more towards the construction of application-specific accelerators to achieve higher performance and lower energy consumption. In order to meet the ever-shortening time constraints, both hardware development and verification tools need to be improved.Chisel, as a hardware construction language, tackles this problem by speeding up the development of digital designs. However, the Chisel infrastructure lacks tools for verification. This paper improves the efficiency of verification in Chisel by proposing methods to support both formal and dynamic verification of digital designs in Scala. It builds on top of ChiselTest, the official testing framework for Chisel. Our work supports functional coverage, constrained random verification, bus functional models, and transaction-level modeling in a verification library named ChiselVerify, while the formal methods are directly integrated into Chisel3.

Bridging Python to Silicon: The SODA Toolchain

Systems performing scientific computing, data analysis, and machine learning tasks have a growing demand for application-specific accelerators that can provide high computational performance while meeting strict size and power requirements. However, the algorithms and applications that need to be accelerated are evolving at a rate that is incompatible with manual design processes based on hardware description languages. Agile hardware design tools based on compiler techniques can help by quickly producing an application-specific integrated circuit (ASIC) accelerator starting from a high-level algorithmic description. We present the software-defined accelerator (SODA) synthesizer, a modular and open-source hardware compiler that provides automated end-to-end synthesis from high-level software frameworks to ASIC implementation, relying on multilevel representations to progressively lower and optimize the input code. Our approach does not require the application developer to write any register-transfer level code, and it is able to reach up to 364 giga floating point operations per second (GFLOPS)/W efficiency (32-bit precision) on typical convolutional neural network operators.

Highly Parallel Multi-FPGA System Compilation from Sequential C/C++ Code in the AWS Cloud

We present a High Level Synthesis compiler that automatically obtains a multi-chip accelerator system from a single-threaded sequential C/C++ application. Invoking the multi-chip accelerator is functionally identical to invoking the single-threaded sequential code the multi-chip accelerator is compiled from. Therefore, software development for using the multi-chip accelerator hardware is simplified, but the multi-chip accelerator can exhibit extremely high parallelism. We have implemented, tested, and verified our push-button system design model on multiple field-programmable gate arrays (FPGAs) of the Amazon Web Services EC2 F1 instances platform, using, as an example, a sequential-natured DES key search application that does not have any DOALL loops and that tries each candidate key in order and stops as soon as a correct key is found. An 8- FPGA accelerator produced by our compiler achieves 44,600 times better performance than an x86 Xeon CPU executing the sequential single-threaded C program the accelerator was compiled from. New features of our compiler system include: an ability to parallelize outer loops with loop-carried control dependences, an ability to pipeline an outer loop without fully unrolling its inner loops, and fully automated deployment, execution and termination of multi-FPGA application-specific accelerators in the AWS cloud, without requiring any manual steps.

Design of Side-Channel-Resistant Power Monitors

In modern computing platforms, power monitors (PwrMons) are employed to deliver online power estimates to support different runtime power-performance optimization methodologies. However, the possibility of setting up a successful side-channel attack by analyzing the power estimates imposes the use of a suitable and systematic approach in the design of such PwrMons. This article proposes a design methodology to automatically identify and implement side-channel-resistant PwrMons at the hardware level, for generic computing platforms. The methodology works by designing a PwrMon for which the switching activity of the signals used to compute the power estimates is not a function of both the secret key and the plaintext/ciphertext values processed by the computing platform. According to the most recent standardized methodologies to assess the side-channel security, our experimental validation leverages both correlation power analysis and <inline-formula> <tex-math notation="LaTeX">$t$ </tex-math></inline-formula>-test analysis considering a general purpose System on Chip executing different cryptographic primitives and an application-specific accelerator implementing the AES-128 algorithm. Our results confirm the impossibility of retrieving the secret key from the power estimates provided by our side-channel-resistant PwrMon. Considering several temporal resolutions, we highlight an accuracy error of the power estimates limited to less than 2.7&#x0025;, as well as an average area and power overheads for the protected PwrMons lower than 6&#x0025; and 5&#x0025;, respectively. To this end, the proposed methodology is able to deliver a side-channel-resistant PwrMon within state-of-the-art accuracy error and overheads.

STT-MRAM-Based Multicontext FPGA for Multithreading Computing Environment

The demand for high-performance computing and rapidly increasing power consumption has increased the necessity for application-specific accelerators. In the datacenter and mobile system, more applications are increasingly relying on accelerators. Field-programmable gate arrays (FPGAs) emerge as a good candidate because they have high programmability and power efficiency. As the number of applications requiring acceleration increases, there is huge demand for FPGAs that support multiple contexts. Previous FPGA designs that support multicontext have various shortcomings such as volatility, poor power efficiency, large performance, area, and reconfiguration overhead. In this article, we propose a spin-transfer torque magnetic RAM (STT-MRAM)-based nonvolatile multicontext FPGA (NVMC-FPGA) that overcomes these shortcomings. We introduce the NVMC-FPGA architecture and operation modes that take advantage of nonvolatility and support multicontext. We also develop the multicontext-aware FPGA computer aided design flow to make the most of the NVMC-FPGA. Compared to the conventional SRAM-based FPGA, when eight identical circuits are mapped, the NVMC-FPGA improves the performance by 15.3% on average and reduces the power consumption by 11.2%–80.7%, depending on the number of simultaneously activated circuits. Moreover, when eight different circuits are mapped, the NVMC-FPGA improves the performance by 58.5% on average and reduces the power consumption by 6.2%–63.3%, depending on the number of simultaneously activated circuits.

Mapping Processing Elements of Custom Virtual CGRAs onto Reconfigurable Partitions

FPGAs can provide application-specific acceleration for computationally demanding tasks. However, they are rarely considered general-purpose platforms due to low productivity of software development and long reconfiguration time. These problems can be mitigated by implementing a coarser overlay atop the FPGA fabric. Combining this approach with partial reconfiguration allows for the modification of individual processing elements (PEs) of the virtual architecture without altering the rest of the system. Module relocation can be used to share implementation details between functionally equivalent PEs that use identical sets of resources, thus eliminating redundant placement and routing runs. Proper floorplanning is crucial for virtual Coarse-Grained Reconfigurable Architectures (CGRAs) with relocatable PEs considering their tendency to use nearest-neighbor connection patterns. It requires solving two problems—finding identical regions in the FPGA fabric and assigning individual partitions to certain locations. This article presents minor improvements of a state-of-the-art solution for the first and proposes a novel technique for solving the other. The proposed automated floorplanner uses modified breadth-first search with direction-based penalties to create initial floorplan consistent with geometry of logical array, then improves the result with 2-opt local optimization. Compared to simulated annealing solutions, the proposed approach allows for the reduction in the floorplanning time by two to three orders of magnitude without compromising the quality of the results.

Domain-specific Hybrid Mapping for Energy-efficient Baseband Processing in Wireless Networks

Advancing telecommunication standards continuously push for larger bandwidths, lower latencies, and faster data rates. The receiver baseband unit not only has to deal with a huge number of users expecting connectivity but also with a high workload heterogeneity. As a consequence of the required flexibility, baseband processing has seen a trend towards software implementations in cloud Radio Access Networks (cRANs). The flexibility gained from software implementation comes at the price of impoverished energy efficiency. This paper addresses the trade-off between flexibility and efficiency by proposing a domain-specific hybrid mapping algorithm. Hybrid mapping is an established approach from the model-based design of embedded systems that allows us to retain flexibility while targeting heterogeneous hardware. Depending on the current workload, the runtime system selects the most energy-efficient mapping configuration without violating timing constraints. We leverage the structure of baseband processing, and refine the scheduling methodology, to enable efficient mapping of 100s of tasks at the millisecond granularity, improving upon state-of-the-art hybrid approaches. We validate our approach on an Odroid XU4 and virtual platforms with application-specific accelerators on an open-source prototype. On different LTE workloads, our hybrid approach shows significant improvements both at design time and at runtime. At design-time, mappings of similar quality to those obtained by state-of-the-art methods are generated around four orders of magnitude faster. At runtime, multi-application schedules are computed 37.7% faster than the state-of-the-art without compromising on the quality.

The TaPaSCo Open-Source Toolflow

The integration of FPGA-based accelerators into a complete heterogeneous system is a challenging task faced by many researchers and engineers, especially now that FPGAs enjoy increasing popularity as implementation platforms for efficient, application-specific accelerators for domains such as signal processing, machine learning and intelligent storage. To lighten the burden of system integration from the developers of accelerators, the open-source TaPaSCo framework presented in this work provides an automated toolflow for the construction of heterogeneous many-core architectures from custom processing elements, and a simple, uniform programming interface to utilize spatially distributed, parallel computation on FPGAs. TaPaSCo aims to increase the scalability and portability of FPGA designs through automated design space exploration, greatly simplifying the scaling of hardware designs and facilitating iterative growth and portability across FPGA devices and families. This work describes TaPaSCo with its primary design abstractions and shows how TaPaSCo addresses portability and extensibility of FPGA hardware designs for systems-on-chip. A study of successful projects using TaPaSCo shows its versatility and can serve as inspiration and reference for future users, with more details on the usage of TaPaSCo presented in an in-depth case study and a short overview of the workflow.

A Compute Cache System for Signal Processing Applications

Nowadays, processing systems are constrained by the low efficiency of their memory subsystems. Although memories evolved into faster and more efficient devices through the years, they were still unable to keep up with the computational power offered by processors, i.e., feed the processors with the data they require at the rhythm it is consumed. Consequently, with the advent of Big Data, the need for fetching large amounts of data from memory became the most prominent performance bottleneck. Naturally, several approaches seeking to mitigate this problem have arisen through the years, such as application-specific accelerators and Near Data Processing (NDP) solutions. However, none were capable to offer a satisfactory general-purpose solution without imposing rather limiting constraints. For instance, NDP solutions often require the programmer to have low-level knowledge of how data is physically stored in memory. In this paper, we propose an alternative mechanism that operates at the cache level, leveraging both proximity to the data and the parallelism enabled by accessing an entire cache line per cycle. We detail the internal architecture of the Cache Compute System (CCS) and demonstrate its integration with a conventional high-performance ARM Cortex-A53 Central Processing Unit (CPU). Furthermore, we assess the performance benefits of the novel CCS using an extensive set of microbenchmarks as well as six kernels widely used in the context of Convolutional Neural Networks (CNNs) and clustering algorithms. Results show that the CCS provides performance improvements ranging from 3.9× to 40.6× regarding the six tested kernels.

APAS: Application-Specific Accelerators for RLWE-Based Homomorphic Linear Transformations

Recently, the application of multi-party secure computing schemes based on homomorphic encryption in the field of machine learning attracts attentions across the research fields. Previous studies have demonstrated that secure protocols adopting packed additive homomorphic encryption (PAHE) schemes based on the ring learning with errors (RLWE) problem exhibit significant practical merits, and are particularly promising in enabling efficient secure inference in machine-learning-as-a-service applications. In this work, we introduce a new technique for performing homomorphic linear transformation (HLT) over PAHE ciphertexts. Using the proposed HLT technique, homomorphic convolutions and inner products can be executed without the use of number theoretic transform and the rotate-and-add algorithms that were proposed in existing works. To maximize the efficiency of the HLT technique, we propose APAS, a hardware-software co-design framework consisting of approximate arithmetic units for the hardware acceleration of HLT. In the experiments, we use actual neural network architectures as benchmarks to show that APAS can improve the computational and communicational efficiency of homomorphic convolution by $8\times $ and $3\times $ , respectively, with an energy reduction of up to $26\times $ as compared to the ASIC implementations of existing methods.

Memory-Augmented Neural Networks on FPGA for Real-Time and Energy-Efficient Question Answering

Memory-augmented neural networks (MANNs) were introduced to handle long-term dependent data efficiently. MANNs have shown promising results in question answering (QA) tasks that require holding contexts for answering a given question. As demands for QA on edge devices have increased, the utilization of MANNs in resource-constrained environments has become important. To achieve fast and energy-efficient inference of MANNs, we can exploit application-specific hardware accelerators on field-programmable gate arrays (FPGAs). Although several accelerators for conventional deep neural networks have been designed, it is difficult to efficiently utilize the accelerators with MANNs due to different requirements. In addition, characteristics of QA tasks should be considered for further improving the efficiency of inference on the accelerators. To address the aforementioned issues, we propose an inference accelerator of MANNs on FPGA. To fully utilize the proposed accelerator, we introduce fast inference methods considering the features of QA tasks. To evaluate our proposed approach, we implemented the proposed architecture on an FPGA and measured the execution time and energy consumption for the bAbI data set. According to our thorough experiments, the proposed methods improved speed and energy efficiency of the inference of MANNs up to about 25.6 and 28.4 times, respectively, compared with those of CPU.

A conceptual framework for quantum accelerated automated design optimization

The development of practical quantum computers that can be used to solve real-world problems is in full swing driven by the ambitious expectation that quantum supremacy will be able to outperform classical super-computers. Like with any emerging compute technology, it needs early adopters in the scientific computing community to identify problems of practical interest that are suitable as proof-of-concept applications and to revise existing solution strategies and develop new ones that exploit the capabilities of the novel compute hardware.In this article we describe a conceptual framework for reducing the computational complexity of simulation-driven automated design optimization processes, which are nowadays widely used in computer-aided product development, by exploiting quantum supremacy. Our approach is based on the assumption that quantum computers will become part of hybrid high-performance computing platforms and can then be used as application-specific accelerator devices.

Galapagos: A Full Stack Approach to FPGA Integration in the Cloud

Field-programmable gate arrays (FPGAs) have shown to be quite beneficial in the cloud due to their energy-efficient application-specific acceleration. These accelerators have always been difficult to use and at cloud scale, the difficulty of managing these devices scales accordingly. We approach the challenge of managing large FPGA accelerator clusters in the cloud using abstraction layers and a new hardware stack that we call Galapagos. The hardware stack abstracts low-level details while providing flexibility in the amount of low-level access users require to reach their performance needs.

Exploring high-performance processor architecture beyond the exascale

The ever-increasing need for high performance in scientific computation and engineering applications will push high-performance computing beyond the exascale. As an integral part of a supercomputing system, highperformance processors and their architecture designs are crucial in improving system performance. In this paper, three architecture design goals for high-performance processors beyond the exascale are introduced, including effective performance scaling, efficient resource utilization, and adaptation to diverse applications. Then a high-performance many-core processor architecture with scalar processing and application-specific acceleration (Massa) is proposed, which aims to achieve the above three goals by employing the techniques of distributed computational resources and application-customized hardware. Finally, some future research directions regarding the Massa architecture are discussed.

Template-Based Parameterized Synthesis of Uniform Instruction-Level Abstractions for SoC Verification

Modern system-on-chip (SoC) designs comprise programmable cores, application-specific accelerators, and I/O devices. Accelerators are controlled by software/firmware and functionality is implemented by this combination of programmable cores, firmware, and accelerators. Verification of such SoCs is challenging, especially for system-level properties maintained by a combination of firmware and hardware. Attempting to formally verify the full SoC design with both firmware and hardware is not scalable, while separate verification can miss bugs. A general technique for scalable system-level verification is to construct an abstraction of SoC hardware and verify firmware/software using it. There are two challenges in applying this technique in practice. Constructing the abstraction to capture required details and interactions is error-prone and time-consuming. The second is ensuring abstraction correctness so that properties proven with it are valid. This paper introduces a methodology for SoC design and verification based on the synthesis of instruction-level abstractions (ILAs). The ILA is an abstraction of SoC hardware which models updates to firmware-visible state at the granularity of instructions. For hardware accelerators, the ILA is analogous to the instruction-set architecture definition for programmable processors and enables scalable verification of firmware interacting with hardware accelerators. To alleviate the disadvantages of manual construction of abstractions, we introduce two algorithms for synthesis of ILAs from partial description called templates. We then show how the ILA can be verified to be correct. We evaluate the methodology using a small SoC design consisting of the 8051 microcontroller and two cryptographic accelerators. The methodology uncovered 15 bugs.

A Flexible Datacenter Simulator

The current data volume and processing requirements in a modern datacenter outgrow the performance increase offered by CPU technology scaling. In light of this trend, datacenter operators have started exploiting application-specific accelerators in order to provide the performance that is needed, without exceeding power consumption constraints. The resulting heterogeneity of resources raises the question of what type of datacenter composition is most useful and under what circumstances. However, as the resulting complexity makes the testing on real-life datacenters difficult or even prohibitive, the use of simulation tools allow datacenter planners to estimate the results of their design decisions. We have developed Het-Sim, a software simulator based on the theoretical modeling of the datacenter, its components, expected workloads, and its possible deployments. Het-Sim takes as inputs a workload and a deployment description and outputs performance metrics of interest, such as task or job latency and resource utilization. In Het-Sim one can configure several parameters, including how tasks are allocated to nodes, and estimations of how fast they execute on different accelerators. Het-Sim can be used to explore how different deployments respond to different kinds of workloads, thus allowing one to determine how to best compose a datacenter based on particular workload, performance, or budgeting requirements.

Energy aware synthesis of application kernels through composition of data-paths on a CGRA

Transistor supply voltages no longer scales at the same rate as transistor density and frequency of operation. This has led to the Dark Silicon problem, wherein only a fraction of transistors can operate at maximum frequency and nominal voltage, in order to ensure that the chip functions within the power and thermal budgets. Heterogeneous computing systems which consist of General Purpose Processors (GPPs), Graphic Processing Units (GPUs) and application specific accelerators can provide improved performance while keeping power dissipation at a realistic level. For the accelerators to be effective, they have to be specialized for related classes of application kernels and have to be synthesized from high level specifications. Coarse Grained Reconfigurable has been proposed as accelerators for a variety of application kernels. For CGRAs to be used as accelerators in the Dark Silicon era, a synthesis framework which focuses on optimizing energy efficiency, while achieving the target performance is essential. However, existing compilation techniques for CGRAs focuses on optimizing only for performance, and any reduction in energy is just a side-effect. In this paper we explore synthesizing application kernels expressed as functions, on a coarse grained composable reconfigurable array (CGCRA). The proposed reconfigurable array comprises HyperCells, which are reconfigurable macro-cells that facilitate modeling power and performance in terms of easily measurable parameters. The proposed synthesis approach takes kernels expressed in a functional language, applies a sequence of well known program transformations, explores trade-offs between throughput and energy using the power and performance models, and realizes the kernels on the CGCRA. This approach when used to map a set of signal processing and linear algebra kernels achieves resource utilization varying from 50% to 80%.