PCIe Bus Research Articles

Strega : An HTTP Server for FPGAs

The computer architecture landscape is being reshaped by the new opportunities, challenges, and constraints brought by the cloud. On the one hand, high-level applications profit from specialised hardware to boost their performance and reduce deployment costs. On the other hand, cloud providers maximise the CPU time allocated to client applications by offloading infrastructure tasks to hardware accelerators. While it is well understood how to do this for, e.g., network function virtualisation and protocols such as TCP/IP, support for higher networking layers is still largely missing, limiting the potential of accelerators. In this article, we present Strega , an open source 1 light-weight Hypertext Transfer Protocol (HTTP) server that enables crucial functionality such as FPGA-accelerated functions being called through a RESTful protocol (FPGA-as-a-Function). Our experimental analysis shows that a single Strega node sustains a throughput of 1.7 M HTTP requests per second with an end-to-end latency as low as 16, μs, outperforming nginx running on 32 vCPUs in both metrics, and can even be an alternative to the traditional OpenCL flow over the PCIe bus. Through this work, we pave the way for running microservices directly on FPGAs, bypassing CPU overhead and realising the full potential of FPGA acceleration in distributed cloud applications.

A generic node identification and routing algorithm in a distributed data acquisition platform: D-Matrix

Data acquisition (DAQ) systems are vital components in large-scale physics experiments. For communication, control and tracking purposes, all system components must be unambiguously identified. D-Matrix, as a generic distributed stream processing DAQ platform, accommodates various device connection methods, including networking, PCIe bus, point-to-point optical fiber links, etc. Although there are mature node identification solutions for each connection method individually, the uniform solution applicable to all connection methods remains crucial in a generic DAQ platform. D-Matrix abstracts a unified Multiple Point-to-Point transmission model (MPP model), supporting various physical connection methods and enabling multiple communication channels on a single physical link. Referring to the tree universal address system, a generic automatic node identification algorithm is proposed based on the MPP model. With simple configuration, this algorithm enables automatic node traversal, yielding a routing-based identification result that supports clustering and hierarchical node management requirements common in large-scale physics experiments. This paper explains the details of the algorithm and presents an example of the DAQ system based on the algorithm.

Design of a high performance Digital data AcquIsition SYstem (DAISY) for innovative nuclear experiments

A dedicated digital data acquisition (DAQ) system has been developed for innovative nuclear data measurements, from time-of-flight to irradiation experiments as well as other basic nuclear physics experiments. The DAQ system is based on fast digitizers with 14 bits resolution and 1 GS/s sampling rate for reliable pulse shape discrimination, energy and timing resolution measurements. Thanks to the high throughput provided by the PCIe bus, data can be processed at CPU or co-processor level, thus allowing great flexibility and reliability. The use of an external clock system and a dedicated triggering system guarantee a very precise synchronization with other possible subsystems in any experimental setup. Finally, its modularity allows to build a flexible and scalable system.

Portable low-cost and highly accurate programmable triggering controller for confocal spinning disk image scanning microscope

This work presents a very low-cost and highly accurate alternative way of implementing triggering control for a confocal spinning-disk image scanning microscope (CSD-ISM). Instead of using the previously reported hig-cost field programmable gate array (FPGA), that connects to a computer by an internal PCIe bus, we implemented the controller using a simple, low-cost real-time digital signal controller (DSC) development kit that connects to a computer via an external USB channel. The overall time resolution of the controller is better than 10 nanoseconds. The DSC programming is implemented using the free software included in the development kit and no additional commercial software is required. The operation of the CSD-ISM is performed by a 32-bit 150 MHz processor using the software environment Micro-Manager, a popular open-source platform for microscopy. Since software is used to calculate all the necessary times, the output signal is almost limitless.

Cross-VM Covert- and Side-Channel Attacks in Cloud FPGAs

The availability of FPGAs in cloud data centers offers rapid, on-demand access to reconfigurable hardware compute resources that users can adapt to their own needs. However, the low-level access to the FPGA hardware and associated resources such as the PCIe bus, SSD drives, or DRAM modules also opens up threats of malicious attackers uploading designs that are able to infer information about other users or about the cloud infrastructure itself. In particular, this work presents a new, fast PCIe-contention-based channel that is able to transmit data between FPGA-accelerated virtual machines (VMs) by modulating the PCIe bus usage. This channel further works with different operating systems and achieves bandwidths reaching 20 kbps with 99% accuracy. This is the first cross-FPGA covert channel demonstrated on commercial clouds and has a bandwidth which is over 2000× larger than prior voltage- or temperature-based cross-board attacks. This article further demonstrates that the PCIe receivers are able to not just receive covert transmissions, but can also perform fine-grained monitoring of the PCIe bus, including detecting when co-located VMs are initialized, even prior to their associated FPGAs being used. Moreover, the proposed mechanism can be used to infer the activities of other users, or even slow down the programming of the co-located FPGAs as well as other data transfers between the host and the FPGA. Beyond leaking information across different virtual machines, the ability to monitor the PCIe bandwidth over hours or days can be used to estimate the data center utilization and map the behavior of the other users. The article also introduces further novel threats in FPGA-accelerated instances, including contention due to network traffic, contention due to shared NVMe SSDs, as well as thermal monitoring to identify FPGA co-location using the DRAM modules attached to the FPGA boards. This is the first work to demonstrate that it is possible to break the separation of privilege in FPGA-accelerated cloud environments, and highlights that defenses for public clouds using FPGAs need to consider PCIe, SSD, and DRAM resources as part of the attack surface that should be protected.

Use of Graphics Processors in Data Compression Algorithms

Query co-processing on graphics processors (GPUs) has become an effective means to improve the performance of main memory databases. However, this co-processing requires the data transfer between the main memory and the GPU memory via a lowbandwidth PCI-E bus. The overhead of such data transfer becomes an important factor, even a bottleneck, for query co-processing performance on the GPU. In this paper, we propose to use compression to alleviate this performance problem.

Design of Data Acquisition System for TOF Detectors in the CEE Experiment

The CSR External-target Experiment(CEE) system under development may reach an average data rate of 2 GB/s after being triggered. In response to the design requirements of the entire spectrometer, a generic stream processing data acquisition architecture D-Matrix is put forward in the article, and a specific scheme of D-Matrix in the Time-of-Flight detector(TOF) subsystem is proposed. The DAQ of the TOF system uses high-speed serial communication methods such as optical fiber and PCIe bus to complete data transmission from Front-End Electronics(FEE) to the servers, and deploys and cascades stream processing nodes with a unified interface to complete event building, command routing, Real-time status monitoring and displaying and data storage tasks. In addition, the DAQ of the TOF system has strong versatility in hardware and logic design, it also can be compatible with other sub-detector systems, and can be further applied to the global system of CEE experiments.

Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems

There is a widening gap between GPU and other components (CPU, PCIe bus and communication network) in heterogeneous parallel system. The gap forces us to orchestrate cooperative execution among these components much more carefully than ever before. By taking the LINPACK benchmark as a case study, this article proposes a fine-grained pipelining algorithm on large-scale CPU-GPU heterogeneous cluster systems. First, we build an algorithmic model that reveals a new approach to GPU-centric and fine-grained pipelining algorithm design. Then, we present four model-driven pipelining algorithms that incrementally squeeze bubbles in the pipeline so that it is occupied by more useful floating-point calculations. The algorithms are implemented on both the AMD and NVIDIA GPU platforms. The finally optimized LINPACK program achieves 107 PFlops on 25, 600 GPUs (70 percent floating-point efficiency). Several insights have been drawn to suggest tradeoff of algorithm design, programming support, and architecture design.

A study on technologies that enable high-speed communication regardless of the interface standard between the host devices and peripheral devices

Recently, I/O interfaces for connecting a computer to peripheral devices have emerged to provide 10Giga services for communication companies. Therefore, Intel has developed Thunderbolt cables, which enable electrical signals and high-speed data transfer to facilitate 10Giga services. Thunderbolt is a type of input–output interface that is used to connect computers and peripherals. It is capable of high-speed input/output(I/O), incurs lower overheads than USB, and supports data, images, and voice simultaneously. As these are very recent standards, they have not been applied to many computers. Moreover, no device exists to use interfaces having a different protocols while differentiating between Thunderbolt and USB devices. Therefore, if both interfaces are mutually incompatible, the user must utilize Thunderbolt or USB devices separately. For this reason, a device that automatically identifies Thunderbolt or USB and automatically adapts to the user’s environment is required. In this study, Thunderbolt MAC and a MAC are alternately connected to each other via an electrical Multiplexer to auto detect link information and connect to Host. Then, based on the determined result, the Thunderbolt to PCIe bridge or the USB to PCIe bridge is electrically connected to the PCIe BUS to which the Multiplexer is connected.

Program package MPGOS: Challenges and solutions during the integration of a large number of independent ODE systems using GPUs

Challenges and efficient solution techniques during the integration of a large number of independent ordinary differential equations (ODEs) using the massively parallel architecture of graphics processing units (GPUs) are presented. One of the main difficulties is the minimisation of the memory transactions through the PCI-E bus between the host (CPU) and the device (GPU) required frequently, for instance, during the calculation of the Lyapunov exponent, winding number or maximum response diagram. The second difficulty is the minimisation of the slow global memory transactions and memory usage by exploiting the memory hierarchy of the GPU architecture. Finally, a good GPU solver has to treat the possible asynchronous features of the ODE systems efficiently; for instance, event detection occurring at distinct time instances or handling the orders of magnitude difference in the required number of time steps of the different ODE systems. The program package MPGOS (written in C++ and CUDA C software environments) can address the aforementioned issues easily via the addition of user-defined functions that must be implemented similarly to the right-hand side of the system; via the possibility of the definition of shared parameters common to all instances of the independent ODE systems; via user-programmable parameters to store only the desired properties of the trajectories; and via an easy was to overlap GPU and CPU computations. This paper focuses on the detailed description of the implementation strategies of the program package.

Breaking the Interaction Wall: A DLPU-Centric Deep Learning Computing System

Due to the broad successes of deep learning, many CPU-centric artificial intelligent computing systems employ specialized devices such as GPUs, FPGAs, and ASICs, which can be named as Deep Learning Processing Units (DLPUs), for processing computation-intensive deep learning tasks. The separation between the scalar control operations mapped on CPUs and the vector computation operations mapped on DLPUs causes the frequent and costly interactions between CPUs and DLPUs, leading to the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Interaction Wall</i> . Moreover, the increasing algorithm complexity and DLPU computation speed would further aggravate the interaction wall substantially. To break the interaction wall, we propose a novel DLPU-centric deep learning computing system consisting of an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">exception-oriented programming (EOP) model</i> and the architectural support of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CPULESS DLPU</i> . The EOP model processes scalar control operations of a deep learning task as exception handlers to maximally avoid stalling the crucial and dominated vector computation operations. Together with the CPULESS DLPU which integrates a scalar processing unit (SPU) for scalar control operations and the parallel processing unit (PPU) for vector computation operations into a fused pipeline, the proposed DLPU-centric system can cost-effectively leverage the EOP model to execute the two kinds of operations simultaneously without disturbing each other. Compared with a state-of-the-art commodity CPU-centric system with discrete V100 GPU via PCIe bus, experimental results show that our DLPU-centric system achieves 10.30× better performance and 92.99 percent energy savings, respectively. Moreover, compared with a CPU-centric version of DLPU system where the SPU serves as the host with integrated PPU, the proposed DLPU-centric system still achieves 15.60 percent better performance from avoided interactions.

FPGA implementation of a run-time configurable NTT-based polynomial multiplication hardware

Multiplication of polynomials of large degrees is the predominant operation in lattice-based cryptosystems in terms of execution time. This motivates the study of its fast and efficient implementations in hardware. Also, applications such as those using homomorphic encryption need to operate with polynomials of different parameter sets. This calls for design of configurable hardware architectures that can support multiplication of polynomials of various degrees and coefficient sizes.In this work, we present the design and an FPGA implementation of a run-time configurable and highly parallelized NTT-based polynomial multiplication architecture, which proves to be effective as an accelerator for lattice-based cryptosystems. The proposed polynomial multiplier can also be used to perform Number Theoretic Transform (NTT) and Inverse NTT (INTT) operations. It supports 6 different parameter sets, which are used in lattice-based homomorphic encryption and/or post-quantum cryptosystems. We also present a hardware/software co-design framework, which provides high-speed communication between the CPU and the FPGA connected by PCIe standard interface provided by the RIFFA driver [1]. For proof of concept, the proposed polynomial multiplier is deployed in this framework to accelerate the decryption operation of Brakerski/Fan-Vercauteren (BFV) homomorphic encryption scheme implemented in Simple Encrypted Arithmetic Library (SEAL), by the Cryptography Research Group at Microsoft Research [2]. In the proposed framework, polynomial multiplication operation in the decryption of the BFV scheme is offloaded to the accelerator in the FPGA via PCIe bus while the rest of operations in the decryption are executed in software running on an off-the-shelf desktop computer. The hardware part of the proposed framework targets Xilinx Virtex-7 FPGA device and the proposed framework achieves the speedup of almost 7 × in latency for the offloaded operations compared to their pure software implementations, excluding I/O overhead.

Toward a hardware man-in-the-middle attack on PCIe bus

The growing need for high rate communication of recent embedded systems is leading to the adoption of the PCIe protocol (Peripheral Component Interconnect express) as an internal data bus. This technology is used in some recent smartphones, and will probably be adopted globally in the next few years. The communication between the processor (in the SoC) and its memory through the PCIe bus represents an important source of information for criminal investigations. In this paper, we present a new attack vector on PCIe based on a hardware Man-in-the-Middle. This system allows real-time data analysis, data-replay, and a copy technique inspired by the shadow-copy principle. Through this one, it is possible to locate, duplicate, and replay sensitive data. The main challenge here is to develop an architecture compliant with PCIe protocol constraints, such as response time, frequency, and throughput, in order to be non-detectable to the communication parts. We designed a proof of concept of an emulator based on a computer with PCIe 3.0 bus and a Stratix 5 FPGA with an endpoint PCIe port as a development target.

ParPaRaw

Parsing is essential for a wide range of use cases, such as stream processing, bulk loading, and in-situ querying of raw data. Yet, the compute-intense step often constitutes a major bottleneck in the data ingestion pipeline, since parsing of inputs that require more involved parsing rules is challenging to parallelise. This work proposes a massively parallel algorithm for parsing delimiter-separated data formats on GPUs. Other than the state-of-the-art, the proposed approach does not require an initial sequential pass over the input to determine a thread's parsing context. That is, how a thread, beginning somewhere in the middle of the input, should interpret a certain symbol (e.g., whether to interpret a comma as a delimiter or as part of a larger string enclosed in double-quotes). Instead of tailoring the approach to a single format, we are able to perform a massively parallel finite state machine (FSM) simulation, which is more flexible and powerful, supporting more expressive parsing rules with general applicability. Achieving a parsing rate of as much as 14.2 GB/s, our experimental evaluation on a GPU with 3 584 cores shows that the presented approach is able to scale to thousands of cores and beyond. With an end-to-end streaming approach, we are able to exploit the full-duplex capabilities of the PCIe bus and hide latency from data transfers. Considering the end-to-end performance, the algorithm parses 4.8 GB in as little as 0.44 seconds, including data transfers.

Hardware Acceleration of AES Cryptographic Algorithm for IPSec

This research considers the offloading of the IPsec packet encryption algorithm into an FPGA by proposing a hardware acceleration of the AES cryptographic algorithm for IPsec. We point out the benefits of relying on HW acceleration in terms of speed and energy efficiency for applications like IPsec. We present the description of the architecture of the proposed solution, the simulation results of our implementation of the AES algorithm in ECB (Electronic Code Book) mode. We also present the integration of the encryption core with the IPsec subsystem through a PCIe bus interface so that the resulting implementation is interoperable with other systems.

Hardware-based approach to guarantee trusted system boot in embedded systems

In recent years, computer systems spread in our daily lives have been experiencing an increasing wave of attacks that disrupt their normal operation or leak sensitive data. Some of them, so-called I/O attacks, are performed by malicious peripherals that perform read or write accesses to DRAM memory through unauthorized DMA (Direct Memory Access) requests during the system boot. During this period the system is particularly vulnerable, which allows hackers to make use of peripherals to perform malicious accesses to DRAM memory. As consequence, it degrades system security. In this context, this paper presents a hardware-based secure-system boot sniffer (SBS) Watchdog to prevent I/O attacks during system boot. The SBS Watchdog is tightly connected to the PCIe communication bus to prevent peripherals from accessing DRAM memory during the BIOS execution and operating system loading, i.e., during the system boot. Compared to existing approaches, the proposed technique is fully transparent to the user, can be applied to any operating system, induces no performance degradation since the SBS Watchdog is turned-off after system boot, and cannot be unintentionally disabled by users during system configuration procedure. The proposed technique was implemented in a commercial FPGA. Practical experiments have demonstrated the high effectiveness of the proposed technique to prevent I/O attacks during system boot.

Speeding up Collective Communications Through Inter-GPU Re-Routing

In order to address the vast needs of disparate domains, computing engines are becoming more sophisticated and complex. A typical high-performance computational engine is composed of several accelerator units, in most cases GPUs, plus one or more CPU controllers. All these components are becoming increasingly interconnected to satisfy bandwidth and latency tolerance demands from modern workloads. Due to these constraints, solutions to efficiently interconnect them or to systematically manage their traffic-such as PCIe v3, NVLink v1 and v2 on the hardware side, and NVIDIA Collective Communication Library (NCCL) and AMD ROCM layer on the software side-are becoming more commonplace inside HPC systems and cloud data centers. However, as the number of accelerators increases, workloads (especially machine learning) might not be able to fully exploit the computational substrate due to inefficient use of hardware interconnects. Such scenarios can lead to performance bottlenecks where high-bandwidth links are not used by the underlying libraries and under-performing links are overused. This work proposes Workload Optimization Through Inter-GPU Re-routing (WOTIR), which consists of enhanced NCCL-based collective primitives that aim to boost bandwidth utilization (through more efficient routing) and reduce communication overhead. WOTIR targets GPUs with no direct NVLink communication path (which leads to PCIe communications) and instead re-routes communication through intermediate GPUs to bridge NVLink segments and avoid PCIe communications. Such method allows the maximum possible utilization of the NVLink bandwidth between the GPUs without routing through the PCIe bus. Using this method, we see a reduction of up to 34 percent in execution time for selected machine learning workloads when non-optimal GPU allocations arise.

Performance assessment of 40 Gbit/s off-the-shelf network cards for virtual network probes in 5G networks

Incoming 5G networks will evolve regarding how they operate due to the use of virtualization technologies. Network functions that are necessary for communication will be virtual and will run on top of commodity servers. Among these functions, it will be essential to deploy monitoring probes, which will provide information regarding how the network is behaving, which will be later analyzed for self-management purposes. However, to date, the network probes have needed to be physical to perform at link-rates in high-speed networks, and it is challenging to deploy them in virtual environments. Thus, it will be necessary to rely on bare-metal accelerators to deal with existing input/output (I/O) performance problems. Next, to control the costs of implementing these virtual network probes, our approach is to leverage the capabilities that current commercial off-the-shelf network cards provide for virtual environments. Specifically, to this end, we have implemented HPCAP40vf, which is a driver that is GPL-licensed and available for download, for network capture in virtual machines. This driver handles the communication with an Intel XL710 40 Gbit/s commercial network card to enable a network monitoring application run within a virtual machine. To store the captured traffic, we have relied on NVMe drives due to their high transference rate, as they are directly connected to the PCIe bus. We have assessed the performance of this approach and compared it with DPDK, in terms of both capturing and storing the network traffic by measuring the achieved data rates. The evaluation has taken into account two virtualization technologies, namely, KVM and Docker, and two access methods to the underlying hardware, namely, VirtIO and PCI passthrough. With this methodology, we have identified bottlenecks and determined the optimal solution in each case to reduce overheads due to virtualization. This approach can also be applied to the development of other performance-hungry virtual network functions. The obtained results demonstrate the feasibility of our proposed approach: when we correctly use the capabilities that current commercial network cards provide, our virtual network probe can monitor at 40 Gbit/s with full packet capture and storage and simultaneously track the traffic among other virtual network functions inside the host and with the external network.

ASW: Accelerating Smith–Waterman Algorithm on Coupled CPU–GPU Architecture

Smith–Waterman algorithm (SW) is a popular dynamic programming algorithm widely used in bioinformatics for local biological sequence alignment. Due to the $$O(n^2)$$ high time and space complexity of SW and growing size of biological data, it is crucial to accelerate SW for high performance. In view of the GPU high efficiency in science computation, many existing studies (e.g., CUDAlign, CUDASW++) speedup SW with GPU. However, the strong data dependency makes SW communication intensive, and the previous works fail to fully leverage the heterogeneous capabilities of the GPU machine for either the neglect of the CPU ability or the low bandwidth of PCI-e. In this paper, we propose ASW, which aims at accelerating SW algorithm with accelerated processing unit (APU), a heterogeneous processor integrates CPU and GPU in a single die and share the same memory. This coupled CPU–GPU architecture is more suitable for frequent data exchanging due to the elimination of PCI-e bus. For the full utilization of both CPU and GPU in APU system, ASW partitions the whole SW matrix into blocks and dynamically dispatches each block to CPU and GPU for the concurrent execution. A DAG-based dynamic scheduling method is presented to dispatch the workload automatically. Moreover, we also design a time cost model to determine the partition granularity in the matrix division phase. We have evaluated ASW on AMD A12 platform and our results show that ASW achieves a good performance of 7.2 GCUPS (gigacells update per second).

A low delay transmission method of multi-channel video based on FPGA

In order to guarantee the fluency of multi-channel video transmission in video monitoring scenarios, we designed a kind of video format conversion method based on FPGA and its DMA scheduling for video data, reduces the overall video transmission delay.In order to sace the time in the conversion process, the parallel ability of FPGA is used to video format conversion. In order to improve the direct memory access (DMA) writing transmission rate of PCIe bus, a DMA scheduling method based on asynchronous command buffer is proposed. The experimental results show that this paper designs a low delay transmission method based on FPGA, which increases the DMA writing transmission rate by 34% compared with the existing method, and then the video overall delay is reduced to 23.6ms.