Software-only Implementation Research Articles

Trireme: Exploration of Hierarchical Multi-level Parallelism for Hardware Acceleration

The design of heterogeneous systems that include domain specific accelerators is a challenging and time-consuming process. While taking into account area constraints, designers must decide which parts of an application to accelerate in hardware and which to leave in software. Moreover, applications in domains such as Extended Reality (XR) offer opportunities for various forms of parallel execution, including loop level, task level, and pipeline parallelism. To assist the design process and expose every possible level of parallelism, we present Trireme , a fully automated tool-chain that explores multiple levels of parallelism and produces domain-specific accelerator designs and configurations that maximize performance, given an area budget. FPGA SoCs were used as target platforms, and Catapult HLS [ 7 ] was used to synthesize RTL using a commercial 12 nm FinFET technology. Experiments on demanding benchmarks from the XR domain revealed a speedup of up to 20×, as well as a speedup of up to 37× for smaller applications, compared to software-only implementations.

AV1 Arithmetic Encoder Design on Open-Source EDA

With the increasing demand for video transmission through the Internet, video coding has become a key technology to allow this market's growth at a reduced cost. Moreover, with the inception of higher video resolutions (e.g., 4K, 8K) and their impact on video size, new video coding standards must tackle this issue to reduce video traffic demand on the global internet infrastructure. The AV1, a recently released royalties-free video coding format created by the Alliance for Open Media (AOMedia), reaches great compression rates but cannot accomplish real-time execution on software-only implementations due to its high complexity. This paper presents and analyzes AE-AV1, a high-performance 4-stage pipelined architecture to accelerate the AV1 arithmetic encoding process (part of the entropy encoder block) and make it capable of real-time execution. For the analysis, this work aims to rely on fully open-source Electronic Design Automation (EDA) tools and Package Design Kits (PDKs).

Using HW/SW Codesign for Deep Neural Network Hardware Accelerator Targeting Low-Resources Embedded Processors

The usage of RISC-based embedded processors, aimed at low cost and low power, is becoming an increasingly popular ecosystem for both hardware and software development. High performance yet low power embedded processors may be attained via the use of hardware acceleration and Instruction Set Architecture (ISA) extension. Efficient mapping of the computational load onto hardware and software resources is a key challenge for performance improvement while still keeping low power and area. Furthermore, exploring performance at an early stage of the design makes this challenge more difficult. Potential hardware accelerators can be identified and extracted from the high-level source code by graph analysis to enumerate common patterns. A scheduling algorithm is used to select an optimized sub-set of accelerators to meet real-time constraints. This paper proposes an efficient hardware/software codesign partitioning methodology applied to high-level programming language at an early stage of the design. The proposed methodology is based on graph analysis. The applied algorithms are presented by a synchronous directed acyclic graph. A constraint-driven method and unique scheduling algorithm are used for graph partitioning to obtain overall speedup and area requirements. The proposed hardware/software partitioning methodology has been evaluated for MLPerf Tiny benchmark. Experimental results demonstrate a speedup of up to 3 orders of magnitude compared to software-only implementation. For example, the resulting runtime for the KWS (Keyword Spotting) software implementation is reduced from 206 sec to only 181ms using the proposed hardware-acceleration approach.

Enfoque de alta granularidad para el procesamiento de paquetes en red para aplicaciones tolerantes a la latencia con CUDA (Corvyd)

En este artículo, estamos interesados en investigar el procesamiento por lotes basado en capas para la inspección de paquetes de red en paralelo. Estudios anteriores de inspección de paquetes basada en GPU [1], [2] y [3] lograron ganancias de rendimiento a través de algunas innovaciones, los más importantes se basaron en la latencia de acceso a la memoria ocultando [4] que no es monopolizado por los sistemas GPU. Los sistemas de procesamiento de paquetes en cualquier hardware también deben usar alguna forma de bypass del kernel para evitar la sobrecarga asociada con las pilas de red de propósito general [4] y [6]. Estos estudios producen un rendimiento dramáticamente mejor que sus predecesores. Por el contrario, el enfoque de este proyecto se limita estrictamente a un conjunto de mejoras algorítmicas. El procesamiento por lotes es común a cualquier implementación paralela de alto rendimiento. Este proyecto propone explorar un procesamiento más granular basado en capas que divide los paquetes a través de múltiples rondas de programación para maximizar la homogeneidad del lote y minimizar la divergencia de la GPU. Esto aumentará significativamente la sobrecarga para procesar un solo paquete, además de aumentar la latencia a medida que se requerirá más almacenamiento en búfer, pero este enfoque tiene el potencial de mejorar el rendimiento en cargas de trabajo de IP altamente erráticas, donde el trabajo previo ha favorecido fuertemente las cargas de trabajo uniformes como ejemplos de PoC minimizados para representar sistemas que probablemente tendrían un bajo rendimiento en el campo [4],[5], y [6].

Universal Verification Platform and Star Simulator for Fast Star Tracker Design.

Developing star trackers quickly is non-trivial. Achieving reproducible results and comparing different algorithms are also open problems. In this sense, this work proposes the use of synthetic star images (a simulated sky), allied with the standardized structure of the Universal Verification Methodology as the base of a design approach. The aim is to organize the project, speed up the development time by providing a standard verification methodology. Future rework is reduced through two methods: a verification platform that us shared under a free software licence; and the layout of Universal Verification Methodology enforces reusability of code through an object-oriented approach. We propose a black-box structure for the verification platform with standard interfaces, and provide examples showing how this approach can be applied to the development of a star tracker for small satellites, targeting a system-on-a-chip design. The same test benches were applied to both early conceptual software-only implementations, and later optimized software-hardware hybrid systems, in a hardware-in-the-loop configuration. This test bench reuse strategy was interesting also to show the regression test capability of the developed platform. Furthermore, the simulator was used to inject specific noise, in order to evaluate the system under some real-world conditions.

The design of scalar AES Instruction Set Extensions for RISC-V

Secure, efficient execution of AES is an essential requirement on most computing platforms. Dedicated Instruction Set Extensions (ISEs) are often included for this purpose. RISC-V is a (relatively) new ISA that lacks such a standardized ISE. We survey the state-of-the-art industrial and academic ISEs for AES, implement and evaluate five different ISEs, one of which is novel. We recommend separate ISEs for 32 and 64-bit base architectures, with measured performance improvements for an AES-128 block encryption of 4x and 10x with a hardware cost of 1.1K and 8.2K gates respectively, when compared to a software-only implementation based on use of T-tables. We also explore how the proposed standard bit-manipulation extension to RISC-V can be harnessed for efficient implementation of AES-GCM. Our work supports the ongoing RISC-V cryptography extension standardisation process.

ReLAccS: A Multilevel Approach to Accelerator Design for Reinforcement Learning on FPGA-Based Systems

Reinforcement learning (RL), specifically Q-learning, with human-like learning abilities to learn from experience without any a priori data, is being increasingly used in embedded systems in the field of control and navigation. However, finding the optimal policy in this approach can be highly compute-intensive, and a software-only implementation may not satisfy the application's timing constraints. To this end, we propose optimization methods at multiple levels of accelerator design for RL. Specifically, at the architecture-level, we exploit the instruction-level parallelism and the spatial parallelism in FPGAs to improve the throughput over state-of-the-art designs by up to 34%. Further, we propose lookup table-level optimizations to reduce the resource utilization and power dissipation of the accelerator. Finally, we propose algorithm-level approximation that can be used for acceleration of Q-learning problems with more states and for reducing the peak power dissipation. We report up to 10× reduction in power dissipation with marginal degradation in quality of results.

Efficient co-design partitioning of WLANs on SoC-based SDRs

Software-defined radio (SDR) is a programmable transceiver with the capability of operating various wireless communication protocols without the need to change or update the hardware. Progress in the SDR field has led to the escalation of protocol development and a wide spectrum of applications, with a greater emphasis on programmability, flexibility, portability, and energy efficiency in cellular, WiFi, and M2M communication. SDR designers intend to simplify the realization of communication protocols while enabling researchers to experiment with prototypes on deployed networks. In this paper, we discuss the HW/SW co-design approach for SDR platforms in the context of wireless communication protocols. We offer a partitioning method of heterogeneous SDR architectures and then discuss the use of Xilinx SDSoC tool for accurate and effective system profiling. To demonstrate our method, we use IEEE 802.11a wireless standard to implement on the Xilinx Zynq-7000 System-on-Chip (SoC). We are able to achieve a significant speedup of 8.7 $$\times$$, compared to a software-only implementation. We also achieved a factor of 1.07 $$\times$$ improvement in terms of power consumption, compared to a hardware-only implementation. Optimization techniques are also adopted for further speedups, and their effectiveness enable us to achieve a speedup of 1.08 $$\times$$, compared to a hardware-only implementation.

High-Performance System-on-Chip-Based Accelerator System for Polynomial Matrix Multiplications

Polynomial matrix computations, such as polynomial matrix multiplication (PMM) and eigenvalue factorization of parahermitian matrices, have played an important role in a growing number of applications, in recent times. However, the computational complexity and expense of such operations impose a profound limit on their applicability. In a recent paper, we introduced a systolic array-based parallel architecture for PMM, which was adequately efficient, but limited in its application. In this paper, we propose a second-generation hardware solution which boasts more versatility, efficiency and scalability compared to our previous design. This is achieved through the design of a highly versatile PMM accelerator which supports polynomial matrices of any size, as a component of the embedded system developed within the Xilinx Zynq-7000 AP SoC. Experimental results demonstrate the efficiency and effectiveness of our novel SoC-based PMM accelerator in the context of subband coding, where maximum speedups of $$85\times $$ and $$33\times $$ are accomplished, without compromising the accuracy, in comparison with two highly optimized and multi-threaded software-only implementations running on a dual-core ARM Cortex-A9 processor and a Intel Core i7-4510U CPU, respectively.

Energy proportional streaming spiking neural network in a reconfigurable system

This paper presents a high-performance architecture for spiking neural networks that optimizes data precision and streaming of configuration data stored in main memory. The neural network is based on the Izhikevich model and mapped to a CPU-FPGA hybrid device using a high-level synthesis flow. The active area of the network is configurable and this feature is used to create an energy proportional system. Voltage and frequency scaling are applied to the processing hardware and memory system to deliver enough processing and memory bandwidth to maintain real-time performance at minimum power and energy levels. The experiments show that the application of voltage and frequency scaling to DDR memory and programmable logic can reduce its energy requirements by up to 77% and 76% respectively. The performance evaluation show that the solution is superior to competing high-performance hardware, while voltage and frequency scaling reduces overall energy requirements to less than 2% of a software-only implementation at the same level of performance.

Application of Ethernet Powerlink for Communication in a Linux RTAI Open CNC system

In computerized numerical control (CNC) systems, the communication bus between the controller and axis servo drives must offer high bandwidth, noise immunity, and time determinism. More and more CNC systems use real-time Ethernet protocols such as Ethernet Powerlink (EPL). Many modern controllers are closed costly hardware-based solutions. In this paper, the implementation of EPL communication bus in a PC-based CNC system is presented. The CNC system includes a PC, a software CNC controller running under Linux Real-Time Application Interface real-time operating system and servo drives communicating via EPL. The EPL stack was implemented as a real-time kernel module. Due to software-only implementation, this system is a cost-effective solution for a broad range of applications in machine control. All software systems are based on GNU General Public License or Berkeley Software Distribution licenses. Necessary modifications to the EPL stack, Linux configurations, computer basic input/output system, and motherboard configurations were presented. Experimental results of EPL communication cycle jitter on three different PCs were presented. The results confirm good performance of the presented system.

An embedded implementation of the Generalized Predictive Control algorithm applied to automotive active suspension systems

The Generalized Predictive Control (GPC) algorithm relies on the solution of an optimization problem at every sampling period. Profiling shows that matrix operations consume the largest portion of the computation requirements of the algorithm. This paper presents an embedded real-time implementation of the GPC algorithm, called GPC-on-Chip, based on the state-of-the-art Customizable Advanced Processor (CAP9™) technology from Atmel®, targeting automotive active suspension systems. Our system utilizes a systolic-array based matrix co-processor in order to accelerate matrix operations. The proposed embedded system is designed to fit within the proposed platform while meeting tight real-time constraints imposed by automotive active suspension systems. In order to check the applicability of the proposed system-on-chip, it is profiled against a wide variety of GPC tuning parameters and compared against the software-only implementation. An average speedup of approximately 10× is achieved.

Experimental Characterization of Uncertainty Sources in a Software-Only Synchronization System

Over the past few decades, centralized computing systems have been replaced by decentralized systems, consisting of simple nodes interconnected by a communication network. The time reference of the distributed nodes must be synchronized in order to be able to coordinate the operation or compare the data collected by the different nodes. Several synchronization protocols have been developed to be used instead of global positioning system or dedicated synchronization systems. For example, the network time protocol (NTP) is a very popular synchronization protocol used to synchronize computers over wide area networks, like the Internet. In addition, IEEE 1588, a synchronization protocol dedicated to high-performance synchronization in local networks, was recently established. However, submicroseconds synchronization can be obtained only using dedicated hardware devices, thereby increasing the cost of each node. For these reasons, PTP software-only implementations are quite common in real systems, with a resulting synchronization uncertainty varying from a few to hundreds of microseconds and quite burdensome to estimate or measure. The work presented in this paper focuses on the analysis of the major uncertainty contributions in a software-only implementation. Particularly, a careful analysis of timestamp mechanism and time management in a PC platform is carried out. In addition, a method for the experimental evaluation of uncertainty contributions is proposed. A test case based on a software implementation of the IEEE 1588 (the so-called PTPd) is presented. The experimental tests presented in the paper highlight that the main uncertainty source of a software-only synchronization approach is the timestamp method. Timestamping accuracy can be affected by the computational load of the node itself: in normal conditions, the maximum uncertainty introduced by the timestamp mechanism is in the order of 10 , but in case of high computational load, it can raise up to 224 μs.

RCDC

Providing deterministic execution significantly simplifies the debugging, testing, replication, and deployment of multithreaded programs. Recent work has developed deterministic multiprocessor architectures as well as compiler and runtime systems that enforce determinism in current hardware. Such work has incidentally imposed strong memory-ordering properties. Historically, memory ordering has been relaxed in favor of higher performance in shared memory multiprocessors and, interestingly, determinism exacerbates the cost of strong memory ordering. Consequently, we argue that relaxed memory ordering is vital to achieving faster deterministic execution. This paper introduces RCDC, a deterministic multiprocessor architecture that takes advantage of relaxed memory orderings to provide high-performance deterministic execution with low hardware complexity. RCDC has two key innovations: a hybrid HW/SW approach to enforcing determinism; and a new deterministic execution strategy that leverages data-race-free-based memory models (e.g., the models for Java and C++) to improve performance and scalability without sacrificing determinism, even in the presence of races. In our hybrid HW/SW approach, the only hardware mechanisms required are software-controlled store buffering and support for precise instruction counting; we do not require speculation. A runtime system uses these mechanisms to enforce determinism for arbitrary programs. We evaluate RCDC using PARSEC benchmarks and show that relaxing memory ordering leads to performance and scalability close to nondeterministic execution without requiring any form of speculation. We also compare our new execution strategy to one based on TSO (total-store-ordering) and show that some applications benefit significantly from the extra relaxation. We also evaluate a software-only implementation of our new deterministic execution strategy.

RCDC

Providing deterministic execution significantly simplifies the debugging, testing, replication, and deployment of multithreaded programs. Recent work has developed deterministic multiprocessor architectures as well as compiler and runtime systems that enforce determinism in current hardware. Such work has incidentally imposed strong memory-ordering properties. Historically, memory ordering has been relaxed in favor of higher performance in shared memory multiprocessors and, interestingly, determinism exacerbates the cost of strong memory ordering. Consequently, we argue that relaxed memory ordering is vital to achieving faster deterministic execution. This paper introduces RCDC, a deterministic multiprocessor architecture that takes advantage of relaxed memory orderings to provide high-performance deterministic execution with low hardware complexity. RCDC has two key innovations: a hybrid HW/SW approach to enforcing determinism; and a new deterministic execution strategy that leverages data-race-free-based memory models (e.g., the models for Java and C++) to improve performance and scalability without sacrificing determinism, even in the presence of races. In our hybrid HW/SW approach, the only hardware mechanisms required are software-controlled store buffering and support for precise instruction counting; we do not require speculation. A runtime system uses these mechanisms to enforce determinism for arbitrary programs. We evaluate RCDC using PARSEC benchmarks and show that relaxing memory ordering leads to performance and scalability close to nondeterministic execution without requiring any form of speculation. We also compare our new execution strategy to one based on TSO (total-store-ordering) and show that some applications benefit significantly from the extra relaxation. We also evaluate a software-only implementation of our new deterministic execution strategy.

Watermarking of Raw Digital Images in Camera Firmware

In this article we investigate ‘real-time’ watermarking of single-sensor digital camera images (often called ‘raw’ images) and blind watermark detection in demosaicked images. We describe the software-only implementation of simple additive spread-spectrum embedding in the firmware of a digital camera. For blind watermark detection, we develop a scheme which adaptively combines the polyphase components of the demosaicked image, taking advantage of the interpolated image structure. Experimental results show the benefits of the novel detection approach for several demosaicking techniques.

Scaling analysis of a neocortex inspired cognitive model on the Cray XD1

This paper presents the implementation and scaling of a neocortex inspired cognitive model on a Cray XD1. Both software and reconfigurable logic based FPGA implementations of the model are examined. This model belongs to a new class of biologically inspired cognitive models. Large scale versions of these models have the potential for significantly stronger inference capabilities than current conventional computing systems. These models have large amounts of parallelism and simple computations, thus allowing highly efficient hardware implementations. As a result, hardware-acceleration of these models can produce significant speedups over fully software implementations. Parallel software and hardware-accelerated implementations of such a model are investigated for networks of varying complexity. A scaling analysis of these networks is presented and utilized to estimate the throughput of both hardware-accelerated and software implementations of larger networks that utilize the full resources of the Cray XD1. Our results indicate that hardware-acceleration can provide average throughput gains of 75 times over software-only implementations of the networks we examined on this system.

FPGA acceleration of a quantum Monte Carlo application

Quantum Monte Carlo methods enable us to determine the ground-state properties of atomic or molecular clusters. Here, we present a reconfigurable computing architecture using Field Programmable Gate Arrays (FPGAs) to accelerate two computationally intensive kernels of a Quantum Monte Carlo (QMC) application applied to N-body systems. We focus on two key kernels of the QMC application: acceleration of potential energy and wave function calculations. We compare the performance of our application on two reconfigurable platforms. Firstly, we use a dual-processor 2.4 GHz Intel Xeon augmented with two reconfigurable development boards consisting of Xilinx Virtex-II Pro FPGAs. Using this platform, we achieve a speedup of 3× over a software-only implementation. Following this, the chemistry application is ported to the Cray XD1 supercomputer equipped with Xilinx Virtex-II Pro and Virtex-4 FPGAs. The hardware-accelerated application on one node of the high performance system equipped with a single Virtex-4 FPGA yields a speedup of approximately 25× over the serial reference code running on one node of the dual-processor dual-core 2.2 GHz AMD Opteron. This speedup is mainly attributed to the use of pipelining, the use of fixed-point arithmetic for all calculations and the fine-grained parallelism using FPGAs. We can further enhance the performance by operating multiple instances of our design in parallel.

FPGA-based real-time 3D image preprocessing for image-guided medical interventions

Minimally invasive image-guided interventions (IGIs) are time and cost efficient, minimize unintended damage to healthy tissue, and lead to faster patient recovery. One emerging trend in IGI workflow is to use volumetric imaging modalities such as low-dose computed tomography (CT) and 3D ultrasound to provide real-time, accurate anatomical information intraoperatively. These intraoperative images, however, are often characterized by quantum (in low-dose CT) or speckle (in ultrasound) noise and must be enhanced prior to any advanced image processing. Anisotropic diffusion filtering and median filtering have been shown to be effective in enhancing and improving the visual quality of these images. However, achieving real-time performance, as required by IGIs, using software-only implementations is challenging because of the sheer size of the images and the arithmetic complexity of the filtering operations. We present a field-programmable gate array-based reconfigurable architecture for real-time preprocessing of intraoperative 3D images. The proposed architecture provides programmable kernels for 3D anisotropic diffusion filtering and 3D median filtering within the same framework. The implementation of this architecture using an Altera Stratix-II device achieved a voxel processing rate close to 200 MHz, which enables the use of these processing techniques in the IGI workflow prior to advanced operations such as segmentation, registration, and visualization.

A UNIFIED APPROACH TO REAL-TIME, MULTI-RESOLUTION, MULTI-BASELINE 2D VIEW SYNTHESIS AND 3D DEPTH ESTIMATION USING COMMODITY GRAPHICS HARDWARE

We present a new method for using commodity graphics hardware to achieve real-time, on-line, 2D view synthesis or 3D depth estimation from two or more calibrated cameras. Our method combines a 3D plane-sweeping approach with 2D multi-resolution color consistency tests. We project camera imagery onto each plane, compute measures of color consistency throughout the plane at multiple resolutions, and then choose the color or depth (corresponding plane) that is most consistent. The key to achieving real-time performance is our use of the advanced features included with recent commodity computer graphics hardware to implement the computations simultaneously (in parallel) across all reference image pixels on a plane. Our method is relatively simple to implement, and flexible in term of the number and placement of cameras. With two cameras and an NVIDIA GeForce4 graphics card we can achieve 50–70 M disparity evaluations per second, including image download and read-back overhead. This performance matches the fastest available commercial software-only implementation of correlation-based stereo algorithms, while freeing up the CPU for other uses.