Custom Accelerators Research Articles

A Parallel Connected Component Labeling Architecture for Heterogeneous Systems-on-Chip

Connected component labeling is one of the most important processes for image analysis, image understanding, pattern recognition, and computer vision. It performs inherently sequential operations to scan a binary input image and to assign a unique label to all pixels of each object. This paper presents a novel hardware-oriented labeling approach able to process input pixels in parallel, thus speeding up the labeling task with respect to state-of-the-art competitors. For purposes of comparison with existing designs, several hardware implementations are characterized for different image sizes and realization platforms. The obtained results demonstrate that frame rates and resource efficiency significantly higher than existing counterparts are achieved. The proposed hardware architecture is purposely designed to comply with the fourth generation of the advanced extensible interface (AXI4) protocol and to store intermediate and final outputs within an off-chip memory. Therefore, it can be directly integrated as a custom accelerator in virtually any modern heterogeneous embedded system-on-chip (SoC). As an example, when integrated within the Xilinx Zynq-7000 X C7Z020 SoC, the novel design processes more than 1.9 pixels per clock cycle, thus furnishing more than 30 2k × 2k labeled frames per second by using 3688 Look-Up Tables (LUTs), 1415 Flip Flops (FFs), and 10 kb of on-chip memory.

Fast and Efficient Convolutional Accelerator for Edge Computing

Convolutional neural networks (CNNs) are a vital approach in machine learning. However, their high complexity and energy consumption make them challenging to embed in mobile applications at the edge requiring real-time processes such as smart phones. In order to meet the real-time constraint of edge devices, recently proposed custom hardware CNN accelerators have exploited parallel processing elements (PEs) to increase throughput. However, this straightforward parallelization of PEs and high memory bandwidth require high data movement, leading to large energy consumption. As a result, only a certain number of PEs can be instantiated when designing bandwidth-limited custom accelerators targeting edge devices. While most bandwidth-limited designs claim a peak performance of a few hundred giga operations per second, their average runtime performance is substantially lower than their roofline when applied to state-of-the-art CNNs such as AlexNet, VGGNet and ResNet, as a result of low resource utilization and arithmetic intensity. In this work, we propose a zero-activation-skipping convolutional accelerator (ZASCA) that avoids noncontributory multiplications with zero-valued activations. ZASCA employs a dataflow that minimizes the gap between its average and peak performances while maximizing its arithmetic intensity for both sparse and dense representations of activations, targeting the bandwidth-limited edge computing scenario. More precisely, ZASCA achieves a performance efficiency of up to 94 percent over a set of state-of-the-art CNNs for image classification with dense representation where the performance efficiency is the ratio between the average runtime performance and the peak performance. Using its zero-skipping feature, ZASCA can further improve the performance efficiency of the state-of-the-art CNNs by up to 1.9× depending on the sparsity degree of activations. The implementation results in 65-nm TSMC CMOS technology show that, compared to the most energy-efficient accelerator, ZASCA can process convolutions from 5.5× to 17.5× faster, and is between 2.1× and 4.5× more energy efficient while occupying 2.1× less silicon area.

A framework to generate domain-specific manycore architectures from dataflow programs

In the last 15 years we have seen, as a response to power and thermal limits for current chip technologies, an explosion in the use of multiple and even many computer cores on a single chip. But now, to further improve performance and energy efficiency, when there are potentially hundreds of computing cores on a chip, we see a need for a specialization of individual cores and the development of heterogeneous manycore computer architectures.However, developing such heterogeneous architectures is a significant challenge. Therefore, we propose a design method to generate domain specific manycore architectures based on RISC-V instruction set architecture and automate the main steps of this method with software tools. The design method allows generation of manycore architectures with different configurations including core augmentation through instruction extensions and custom accelerators. The method starts from developing applications in a high-level dataflow language and ends by generating synthesizable Verilog code and cycle accurate emulator for the generated architecture.We evaluate the design method and the software tools by generating several architectures specialized for two different applications and measure their performance and hardware resource usages. Our results show that the design method can be used to generate specialized manycore architectures targeting applications from different domains. The specialized architectures show at least 3 to 4 times better performance than the general purpose counterparts. In certain cases, replacing general purpose components with specialized components saves hardware resources. Automating the method increases the speed of architecture development and facilitates the design space exploration of manycore architectures.

Real-time audio signal processing using system-on-chip field programmable gate arrays

System-on-Chip (SoC) Field Programmable Gate Arrays (FPGAs) are ideal for real-time signal processing due to their low, deterministic latency and high performance. To showcase the utility of our open FPGA computational platform for real-time audio signal processing and computational modeling, several applications have been implemented. We have ported the openMHA hearing aid software [1] to our platform to show that pre-existing audio processing software can be implemented in SoC FPGAs by making external audio interfaces show up as a sound card. To highlight the ability to perform real-time computational modeling on our performance platform, we are implementing a real-time version of Laurel Carney’s auditory-nerve model [2] running in its own custom accelerator in the FPGA fabric. To illustrate the ability to develop DSP algorithms in MathWork’s Simulink and then implement them in the FPGA fabric we have taken several algorithms from Issa Panahi’s group [3] to show both frame-based processing (noise reduction) and sample-based processing (dynamic range compression). Finally, we show that the platform can be used to visualize audio signals using a real-time spectrogram where FFTs are computed in the FPGA fabric. [1] www.openmha.org. [2] JASA 126, 2390–2412. [3] www.utdallas.edu/ssprl/hearing-aid-project/.

Human Lumbar Spine Responses from Vertical Loading: Ranking of Forces Via Brier Score Metrics and Injury Risk Curves.

This study was conducted to quantify the human tolerance from inferior to superior impacts to whole lumbar spinal columns excised from 43 post mortem human subjects. The specimens were fixed at the ends, aligned in a consistent seated posture, load cells were attached to the proximal and distal ends of the fixation, and the impact was applied using a custom accelerator device. Pretest X-rays and computed tomography (CT) scans, prepositioned X-rays, and posttest X-rays, CT scans and dissection data were used to identify injuries. Right, left, and interval censoring processes were used for the survival analysis, 16 were right censored, 24 were interval censored, and three were left censored observations. Force-based injury risk curves were developed, and the optimal metric describing the underlying response to injury was identified using the Brier score metric. Material, geometry (disc and body areas), and demographic covariates were included in the analysis. The distal force was found to be optimal metric. The bone mineral density was a significant covariate for distal and proximal forces. Both material and geometrical factors affected the transmitted force in this mode of loading. These quantified data serve as the first set of human lumbar spinal column injury risk curves.

PDES-A

In this article, we present experiences implementing a general Parallel Discrete Event Simulation (PDES) accelerator on a Field Programmable Gate Array (FPGA). The accelerator can be specialized to any particular simulation model by defining the object states and the event handling code, which are then synthesized into a custom accelerator for the given model. The accelerator consists of several event processors that can process events in parallel while maintaining the dependencies between them. Events are automatically sorted by a self-sorting event queue. The accelerator supports optimistic simulation by automatically keeping track of event history and supporting rollbacks. The architecture is limited in scalability locally by the communication and port bandwidth of the different structures. However, it is designed to allow multiple accelerators to be connected to scale up the simulation. We evaluate the design and explore several design trade-offs and optimizations. We show that the accelerator can scale to 64 concurrent event processors relative to the performance of a single event processor. At this point, the scalability becomes limited by contention on the shared structures within the datapath. To alleviate this bottleneck, we also develop a new version of the datapath that partitions the state and event space of the simulation but allows these partitions to share the use of the event processors. The new design substantially reduces contention and improves the performance with 64 processors from 49x to 62x relative to a single processor design. We went through two iterations of the design of PDES-A, first using Verilog and then using Chisel (for the partitioned version of the design). We report in this article on some observations in the differences in prototyping accelerators using these two different languages. PDES-A outperforms the ROSS simulator running on a 12-core Intel Xeon machine by a factor of 3.2x with less than 15% of the power consumption. Our future work includes building multiple interconnected PDES-A cores.

KNN-STUFF: kNN STreaming Unit for Fpgas

This paper presents kNN STreaming Unit For Fpgas (kNN-STUFF), a modular, scalable and efficient Hardware/Software implementation of k-Nearest Neighbors (kNN) classifier targeting System on Chip (SoC) devices. It takes advantage of custom accelerators, implemented on the reconfigurable fabric of the SoC device, to perform most of the classifier's workload, whereas the processor coordinates the accelerators and runs the remaining workload of the kNN algorithm. kNN-STUFF offers a highly flexible framework, where the designer has the possibility to define the number of parallel instances of the classifier and the parallelism within each instance. This capability allows creating the most suitable implementation for a target device of any size. Results show that kNN-STUFF, with 24 accelerators, attains performance improvements up to 67.4×, when compared to an optimized (-O3) software-only implementation of the kNN running on a single core of the ARM Cortex-A9 CPU. Furthermore, its energy efficiency improvements are as high as 50.6×.

Accelerating Board Games Through Hardware/Software Codesign

Board games applications usually offer a great user experience when running on desktop computers. Powerful high-performance processors working without energy restrictions successfully deal with the exploration of large game trees, delivering strong play to satisfy demanding users. However, nowadays, more and more game players are running these games on smartphones and tablets, where the lower computational power and limited power budget yield a much weaker play. Recent systems-on-a-chip include programmable logic tightly coupled with general-purpose processors enabling the inclusion of custom accelerators for any application to improve both performance and energy efficiency. In this paper, we analyze the benefits of partitioning the artificial intelligence of board games into software and hardware. We have chosen as case studies three popular and complex board games, Reversi , Blokus , and Connect6 . The designs analyzed include hardware accelerators for board processing, which improve performance and energy efficiency by an order of magnitude leading to much stronger and battery-aware applications. The results demonstrate that the use of hardware/software codesign to develop board games allows sustaining or even improving the user experience across platforms while keeping power and energy low.

IO and data management for infrastructure as a service FPGA accelerators

We describe the design of a non-operating-system based embedded system to automate the management, reordering, and movement of data produced by FPGA accelerators within data centre environments. In upcoming cloud computing environments, where FPGA acceleration may be leveraged via Infrastructure as a Service (IaaS), end users will no longer have full access to the underlying hardware resources. We envision a partially reconfigurable FPGA region that end-users can access for their custom acceleration needs, and a static “template” region offered by the data centre to manage all Input/Output (IO) data requirements to the FPGA. Thus our low-level software controlled system allows for standard DDR access to off-chip memory, as well as DMA movement of data to and from SATA based SSDs, and access to Ethernet stream links. Two use cases of FPGA accelerators are presented as experimental examples to demonstrate the area and performance costs of integrating our data-management system alongside such accelerators. Comparisons are also made to fully custom data management solutions implemented solely in RTL Verilog to determine the tradeoffs in using our system in regards to development time, area, and performance. We find that for a class of accelerators in which the physical data rate of an IO channel is the limiting bottleneck to accelerator throughput, our solution offers drastically reduced logic development time spent on data management without any associated performance losses in doing so. However, for a class of applications where the IO channel is not the bottle-neck, our solution trades off increased area usage to save on design times and to maintain acceptable system throughput in the face of degraded IO throughput.

Abstract WP125: Accelerated 3d Isotropic High-resolution Multi-contrast Intracranial Vessel Wall MRI For Large Artery Stroke Evaluation

Introduction: Intracranial large-artery atherosclerotic disease (ICAD) is a leading cause of death and morbidity worldwide. Vessel Wall Imaging (VWI) has potential to stratify disease beyond current angiographic methods by identifying plaque morphology and components directly. Plaque component identification requires a multi-contrast black-blood protocol. High isotropic resolution is also required due to small vessels and their complex geometry and therefore typical ICAD VWI protocols require long scan times. Aim: To develop a 3D 0.5 mm isotropic multi-contrast ICAD VWI with scan time of less than 30 minutes. Methods: Experiments were carried out on a Philips Ingenia 3T scanner with 32 channel head coil. Acceleration by K-space undersampling using CUSTOM method and compressed sensing reconstruction using STEP method [1] was used to reduce scan times. The following protocol was optimized on phantoms and volunteers: Survey scan was followed by 3D TOF, T1 weighted VISTA, PD weighted DANTE [2] VISTA, SNAP [3]. After single dose gadolinium contrast injection a post-contrast T1 weighted VISTA is obtained. Protocol parameters were optimized such that total scan time is 25 mins with all scans (except survey) being 0.5mm isotropic. Results: CUSTOM acceleration factors of 4X to 5X provided good image quality. Representative image quality (pre-contrast) is shown in figure 1. Total scan time for the multi-contrast protocol was 30 minutes including patient setup time. Studies on ICAD patients are ongoing with the optimized protocol. Conclusions: Multi-contrast 3D 0.5mm isotropic resolution MRI protocol was developed to scan ICAD patients within 30 minute scan time. Acceleration tailored for VWI allowed reducing scan times from 2 hours to less than 30 minutes thereby providing a clinically usable ICAD VWI multi-contrast protocol.

A Custom Accelerator for Homomorphic Encryption Applications

After the introduction of first fully homomorphic encryption scheme in 2009, numerous research work has been published aiming at making fully homomorphic encryption practical for daily use. The first fully functional scheme and a few others that have been introduced has been proven difficult to be utilized in practical applications, due to efficiency reasons. Here, we propose a custom hardware accelerator, which is optimized for a class of reconfigurable logic, for Lopez-Alt, Tromer and Vaikuntanathan's somewhat homomorphic encryption based schemes. Our design is working as a co-processor which enables the operating system to offload the most compute-heavy operations to this specialized hardware. The core of our design is an efficient hardware implementation of a polynomial multiplier as it is the most compute-heavy operation of our target scheme. The presented architecture can compute the product of very-large polynomials in under 6.25 ms which is 102 times faster than its software implementation. In case of accelerating homomorphic applications; we estimate the per block homomorphic AES as 442 ms which is 28.5 and 17 times faster than the CPU and GPU implementations, respectively. In evaluation of Prince block cipher homomorphically, we estimate the performance as 52 ms which is 66 times faster than the CPU implementation.

Constructive Synthesis of Memory-Intensive Accelerators for FPGA From Nested Loop Kernels

Field-programmable gate arrays are ideal hosts to custom accelerators for signal, image, and data processing but demand manual register transfer level design if high performance and low cost are desired. High-level synthesis reduces this design burden but requires manual design of complex on-chip and off-chip memory architectures, a major limitation in applications such as video processing. This paper presents an approach to resolve this shortcoming. A constructive process is described that can derive such accelerators, including on- and off-chip memory storage from a C description such that a user-defined throughput constraint is met. By employing a novel statement-oriented approach, dataflow intermediate models are derived and used to support simple approaches for on-/off-chip buffer partitioning, derivation of custom on-chip memory hierarchies and architecture transformation to ensure user-defined throughput constraints are met with minimum cost. When applied to accelerators for full search motion estimation, matrix multiplication, Sobel edge detection, and fast Fourier transform, it is shown how real-time performance up to an order of magnitude in advance of existing commercial HLS tools is enabled whilst including all requisite memory infrastructure. Further, optimizations are presented that reduce the on-chip buffer capacity and physical resource cost by up to 96% and 75%, respectively, whilst maintaining real-time performance.

Streaming Elements for FPGA Signal and Image Processing Accelerators

Field-programmable gate array (FPGA) devices boast abundant resources with which custom accelerator components for signal, image, and data processing may be realized; however, realizing high-performance, low-cost accelerators currently demands manual register transfer level design. Software-programmable soft processors have been proposed as a way to reduce this design burden, but they are unable to support performance and cost comparable to custom circuits. This paper proposes a new soft processing approach for FPGA that promises to overcome this barrier. A high-performance, fine-grained streaming processor, known as a streaming accelerator element, is proposed, which realizes accelerators as large-scale custom multicore networks. By adopting a streaming execution approach with advanced program control and memory addressing capabilities, typical program inefficiencies can be almost completely eliminated to enable performance and cost, which are unprecedented among software-programmable solutions. When used to realize accelerators for fast Fourier transform, motion estimation, matrix multiplication, and sobel edge detection, it is shown how the proposed architecture enables real-time performance and with performance and cost comparable with hand-crafted custom circuit accelerators and up to two orders of magnitude beyond existing soft processors.

TBES

This article describes TBES, a software end-to-end environment for synthesizing multitask applications on FPGAs. The implementation follows a template-based approach for creating heterogeneous multiprocessor architectures. Heterogeneity stems from the use of general-purpose processors along with custom accelerators. Experimental results demonstrate substantial speedup for several classes of applications. Furthermore, this work allows for reducing development costs and saving development time for the software architect, the domain expert, and the optimization expert. This work provides a framework to bring together various existing tools and optimisation algorithms. The advantages are manifold: modularity and flexibility, easy customization for best-fit algorithm selection, durability and evolution over time, and legacy preservation including domain experts' know-how. In addition to the use of architecture templates for the overall system, a second contribution lies in using high-level synthesis for promoting exploration of hardware IPs. The domain expert, who best knows which tasks are good candidates for hardware implementation, selects parts of the initial application to be potentially synthesized as dedicated accelerators. As a consequence, the HLS general problem turns into a constrained and more tractable issue, and automation capabilities eliminate the need for tedious and error-prone manual processes during domain space exploration. The automation only takes place once the application has been broken down into concurrent tasks by the designer, who can then drive the synthesis process with a set of parameters provided by TBES to balance tradeoffs between optimization efforts and quality of results. The approach is demonstrated step by step up to FPGA implementations and executions with an MJPEG benchmark and a complex Viola-Jones face detection application. We show that TBES allows one to achieve results with up to 10 times speedup to reduce development times and to widen design space exploration.

Real-time simulation of dynamic vehicle models using a high-performance reconfigurable platform

With the increase in the complexity of models and lack of flexibility offered by the analog computers, coupled with the advancements in digital hardware, the simulation industry has subsequently moved to digital computers and increased usage of programming languages such as C, C++, and MATLAB. However, the reduced time-step required to simulate complex and fast systems imposes a tighter constraint on the time within which the computations have to be performed. The sequential execution of these computations fails to cope with the real-time constraints which further restrict the usefulness of Real-Time Simulation (RTS) in a Virtual Reality (VR) environment. In this paper, we present a methodology for the design and implementation of RTS algorithms, based on the use of Field-Programmable Gate Array (FPGA) technology. We apply our methodology to an 8th order steering valve subsystem of a vehicle with relatively low response time requirements and use the FPGA technology to improve the response time of this model. Our methodology utilizes traditional hardware/software co-design approaches to generate a heterogeneous architecture for an FPGA-based simulator by porting the computationally complex regions to hardware. The hardware design was optimized such that it efficiently utilizes the parallel nature of FPGAs and pipelines the independent operations. Further enhancement was made by building a hardware component library of custom accelerators for common non-linear functions. The library also stores the information about resource utilization, cycle count, and the relative error with different bit-width combinations for these components, which is further used to evaluate different partitioning approaches. In this paper, we illustrate the partitioning of a hardware-based simulator design across dual FPGAs, initiate RTS using a system input from a Hardware-in-the-Loop (HIL) framework, and use these simulation results from our FPGA-based platform to perform response analysis. The total simulation time, which includes the time required to receive the system input over a socket (without HIL), software initialization, hardware computation, and transfer of simulation results back over a socket, shows a speedup of 2 × as compared to a similar setup with no hardware acceleration. The correctness of the simulation output from the hardware has also been validated with the simulated results from the software-only design.

Data-Transfer-Aware Design of an FPGA-Based Heterogeneous Multicore Platform with Custom Accelerators

Data-Transfer-Aware Design of an FPGA-Based Heterogeneous Multicore Platform with Custom Accelerators

Comparison of High Level FPGA Hardware Design for Solving Tri-diagonal Linear Systems

Reconfigurable computing devices can increase the performance of compute intensive algorithms by implementing application specific co-processor architectures. The power cost for this performance gain is often an order of magnitude less than that of modern CPUs and GPUs. Exploiting the potential of reconfigurable devices such as Field-Programmable Gate Arrays (FPGAs) is typically a complex and tedious hardware engineering task. Recently the major FPGA vendors (Altera, and Xilinx) have released their own high-level design tools, which have great potential for rapid development of FPGA based custom accelerators. In this paper, we will evaluate Altera's OpenCL Software Development Kit, and Xilinx's Vivado High Level Sythesis tool. These tools will be compared for their performance, logic utilisation, and ease of development for the test case of a tri-diagonal linear system solver.

Evaluation of an FPGA-Based Heterogeneous Multicore Platform with SIMD/MIMD Custom Accelerators

SUMMARY Heterogeneous multi-core architectures with CPUs andaccelerators attract many attentions since they can achieve power-efficientcomputing in various areas from low-power embedded processing to high-performance computing. Since the optimal architecture is different fromapplication to application, finding the most suitable accelerator is very im-portant. In this paper, we propose an FPGA-based heterogeneous multi-core platform with custom accelerators for power-efficient computing. Us-ing the proposed platform, we evaluate several applications and accelera-tors to identify many key requirements of the applications and propertiesof the accelerators. Such an evaluation is very important to select and op-timize the most suitable accelerator according to the requirements of anapplication to achieve the best performance. key words: heterogeneous multicore processor, FPGA, Multimedia pro-cessing, High-performance-computing 1. Introduction Applications used in low-power embedded processing tohigh performance computing have different tasks such asdata-intensive tasks and control-intensive tasks. Therefore,optimal architecture is different from application to applica-tion. Heterogeneous multicore processing is proposed to ex-ecute applications power-efficiently. It uses different proces-sor cores such as CPU cores and accelerator cores as shownin Fig.1. If the tasks of an application are correctly allocatedto the most suitable processor cores, all the cores work to-gether to increase the overall performances.Examples of low-power heterogeneous multi-core pro-cessors are [1] and [2]. The former has multiple coresof CPUs and ALU arrays. The latter has multiple coresof CPUs, a micro-controller and SIMD (single-instructionmultiple-data) type processors. An example of a hetero-geneous high-performance computing is “Tianhe-1A” [3]which has Intel X5670 CPUs and NVDIA GPUs. Com-mercially available heterogeneous multicore processors arepartially programmable so that a part of the data path andcomputations of processing elements (PEs) can be changedto some extent. However, due to the wide variety of tasksand their different memory requirements, the programma-bility in commercially available processors is not enough toextract sufficient performance. Moreover, the programmingenvironments in various heterogeneous architectures such as

Real-time Simulation of Dynamic Vehicle Models using a High-performance Reconfigurable Platform

A purely software-based approach for Real-Time Simulation (RTS) may have difficulties in meeting real-time constraints for complex physical model simulations. In this paper, we present a methodology for the design and im-plementationofRTS algorithms,basedontheuseof Field-ProgrammableGateArray(FPGA) technologytoimprove the response time of these models. Our methodology utilizes traditional hardware/software co-design approaches to generate a heterogeneous architecture for an FPGA-based simulator. The hardware design was optimized such that it efficiently utilizes the parallel nature of FPGAs and pipelines the independent operations. Further enhancement is obtained through the use of custom accelerators for common non-linear functions. Since the systems we examined had relatively low response time requirements, our approach greatly simplifies the software components by porting the computationally complexregionsto hardware.We illustratethe partitioningofa hardware-based simulator design across dual FPGAs, initiateRTS usinga system input froma Hardware-in-the-Loop (HIL) framework, and use these simulation results from our FPGA-based platform to perform response analysis. The total simulation time, which includes the time required to receive the system input over a socket (without HIL), software initialization, hardware computation, and transferof simulation results backovera socket, showsa speedup of 2× as compared to a simi-lar setup with no hardware acceleration. The correctness of the simulation output from the hardware has also been validated with the simulated results from the software-only design.

Exploiting Heterogeneity for Energy Efficiency in Chip Multiprocessors

Heterogeneous multicores are envisioned to be a promising design paradigm to combat today's challenges of power, memory, and reliability walls that are impeding chip design using deep submicron technology. Future multicores are expected to integrate multiple different cores, including GPGPUs, custom accelerators and configurable cores. In this paper, we introduce an important dimension-technology-using which heterogeneity can be introduced in multicores to improve their energy-performance envelope. Specifically, we analyze the benefits of heterogenous technologies for processor cores and cache subsystems. We discuss two promising device candidates (Tunnel-FET and Magnetic-RAM) for introducing technological diversity in the multicores and analyze their integration in the processor and cache hierarchy in detail. Our analysis shows that introducing such a kind of heterogeneity can significantly enhance the performance and energy behavior of future multicore systems.