SIMD Intrinsics Research Articles

Cross Layer Design Using HW/SW Co-Design and HLS to Accelerate Chaining in Genomic Analysis

DNA sequence analysis is a computationally intensive task. Minimap2 is a state-of-the-art software tool for third-generation sequence analysis workflow. Nearly 50% of Minimap2’s computation time is spent on what is known as the chaining step. In this paper, the chaining step is accelerated using a novel heterogeneous computing system combining an Intel FPGA-based hardware accelerator and a CPU (using HLS for the FPGA and multi-threaded software for the CPU). The system in this paper is capable of handling large realistic workloads, achieves up to ~1.35× performance improvement over the software solution running on an Intel CPU with SIMD intrinsics (Intel’s latest AVX-512), while consuming ~27% less energy. When compared to the software solution running on the CPU without SIMD intrinsics, the system performs ~1.9× faster while consuming ~38% less energy. Importantly, this work also ensures that the accuracy of the output generated is not compromised for the speed-up gained (this work only has an error rate of less than 10-6% compared to 26% in previous FPGA-based chaining step accelerator).

The FastLanes Compression Layout: Decoding > 100 Billion Integers per Second with Scalar Code

The open-source FastLanes project aims to improve big data formats, such as Parquet, ORC and columnar database formats, in multiple ways. In this paper, we significantly accelerate decoding of all common Light-Weight Compression (LWC) schemes: DICT, FOR, DELTA and RLE through better data-parallelism. We do so by re-designing the compression layout using two main ideas: (i) generalizing the value interleaving technique in the basic operation of bit-(un)packing by targeting a virtual 1024-bits SIMD register, (ii) reordering the tuples in all columns of a table in the same Unified Transposed Layout that puts tuple chunks in a common "04261537" order (explained in the paper); allowing for maximum independent work for all possible basic SIMD lane widths: 8, 16, 32, and 64 bits. We address the software development, maintenance and future-proofness challenges of increasing hardware diversity, by defining a virtual 1024-bits instruction set that consists of simple operators supported by all SIMD dialects; and also, importantly, by scalar code. The interleaved and tuple-reordered layout actually makes scalar decoding faster, extracting more data-parallelism from today's wide-issue CPUs. Importantly, the scalar version can be fully auto-vectorized by modern compilers, eliminating technical debt in software caused by platform-specific SIMD intrinsics. Micro-benchmarks on Intel, AMD, Apple and AWS CPUs show that FastLanes accelerates decoding by factors (decoding >40 values per CPU cycle). FastLanes can make queries faster, as compressing the data reduces bandwidth needs, while decoding is almost free.

Compiler Support for an AI-oriented SIMD Extension of a Space Processor

In this on going research paper, we present our work on the compiler support for an AI-oriented SIMD Extension, called SPARROW. The SPARROW hardware design has been developed during a recently defended, awardwinning Master Thesis and is targeting Cobham Gaisler's space processors Leon3 and NOEL-V. We present the compiler support we have included in two compiler toolchains, gcc and llvm as well as a SIMD intrinsics library for easy programmability. Compiler modifications are kept to minimum in order to enable incremental qualification of the toolchains. We present our experience working with the two compilers and performance results for the two compilers on top an FPGA implementation of the target space processor.

Design and Implementation of 2D Convolution on x86/x64 Processors

In this paper, a new method for accelerating the 2D direct Convolution operation on x86/x64 processors is presented. It includes efficient vectorization by using SIMD intrinsics, bit-twiddling optimizations, the optimization of the division operation, multi-threading using OpenMP, register blocking and the shortest possible bit-width value of the intermediate results. The proposed method, which is provided as open-source, is general and can be applied to other processor families too, e.g., Arm. The proposed method has been evaluated on two different multi-core Intel CPUs, by using twenty different image sizes, 8-bit integer computations and the most commonly used kernel sizes (3x3, 5x5, 7x7, 9x9). It achieves from <inline-formula><tex-math notation="LaTeX">$2.8\times$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$40\times$</tex-math></inline-formula> speedup over the Intel IPP library (OpenCV GaussianBlur and Filter2D routines), from <inline-formula><tex-math notation="LaTeX">$105 \times$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$400 \times$</tex-math></inline-formula> speedup over the gemm-based convolution method (by using Intel MKL int8 matrix multiplication routine), and from <inline-formula><tex-math notation="LaTeX">$8.5\times$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$618\times$</tex-math></inline-formula> speedup over the vslsConvExec Intel MKL direct convolution routine. The proposed method is superior as it achieves far fewer arithmetical and load/store instructions.

Improving Performance with SIMD Intrinsics

Computer application development has a long history. With the continuous development of hardware, both the software and the programming languages and frameworks that make application development possible have also changed. As we move forward in time, as more and more powerful computers become available, the level of programming languages has also risen to a higher level. Increasingly high-level APIs and languages have become available, pointers, memory allocations and releases have slowly disappeared from the area of the programming. However, with the transition to high-level languages, the optimization of programs or the possibility of it has been relegated to the background, because some efficiency-enhancing options are not available in high-level languages. This paper therefore delves deeper into the benefits of SSE/AVX based SIMD concurrency capabilities that are “forgotten” in everyday programming. The presented tests will show that there are reserves in the CPU that can be used to push certain performance limits further away.

The Design and Experiments of A SID-Based Power-Aware Simulator for Embedded Multicore Systems

Embedded multicore systems are playing increasingly important roles in the design of consumer electronics. The objective of such systems is to optimize both performance and power characteristics of mobile devices. However, currently there are no power metrics supporting popular application design platforms (such as SID) that application developers use to develop their applications. This hinders the ability of application developers to optimize power consumption. In this article we present the design and experiments of a SID-based power-aware simulation framework for embedded multicore systems. The proposed power estimation flow includes two phases: IP-level power modeling and power-aware system simulation. The first phase employs PowerMixer IP to construct the power model for the processor IP and other major IPs, while the second phase involves a power abstract interpretation method for summarizing the simulation trace, then, with a CPE module, estimating the power consumption based on the summarized trace information and the input of IP power models. In addition, a Manager component is devised to map each digital signal processor (DSP) component to a host thread and maintain the access to shared resources. The aim is to maintain the simulation performance as the number of simulated DSP components increases. A power-profiling API is also supported that developers of embedded software can use to tune the granularity of power-profiling for a specific code section of the target application. We demonstrate via case studies and experiments how application developers can use our SID-based power simulator for optimizing the power consumption of their applications. We characterize the power consumption of DSP applications with the DSPstone benchmark and discuss how compiler optimization levels with SIMD intrinsics influence the performance and power consumption. A histogram application and an augmented-reality application based on human-face-based RMS (recognition, mining, and synthesis) application are deployed as running examples on multicore systems to demonstrate how our power simulator can be used by developers in the optimization process to illustrate different views of power dissipations of applications.

Cross-Loop Optimization of Arithmetic Intensity for Finite Element Local Assembly

We study and systematically evaluate a class of composable code transformations that improve arithmetic intensity in local assembly operations, which represent a significant fraction of the execution time in finite element methods. Their performance optimization is indeed a challenging issue. Even though affine loop nests are generally present, the short trip counts and the complexity of mathematical expressions, which vary among different problems, make it hard to determine an optimal sequence of successful transformations. Our investigation has resulted in the implementation of a compiler (called COFFEE) for local assembly kernels, fully integrated with a framework for developing finite element methods. The compiler manipulates abstract syntax trees generated from a domain-specific language by introducing domain-aware optimizations for instruction-level parallelism and register locality. Eventually, it produces C code including vector SIMD intrinsics. Experiments using a range of real-world finite element problems of increasing complexity show that significant performance improvement is achieved. The generality of the approach and the applicability of the proposed code transformations to other domains is also discussed.

GPU Acceleration of Runge-Kutta Integrators

We consider the use of commodity graphics processing units (GPUs) for the common task of numerically integrating ordinary differential equations (ODEs), achieving speedups of up to 115-fold over comparable serial CPU implementations, and 15-fold over multithreaded CPU code with SIMD intrinsics. Using Lorenz '96 models as a case study, single and double precision benchmarks are established for both the widely used DOPRI5 method and computationally tailored low-storage RK4(3)5[2R+]C. A range of configurations are assessed on each, including multithreading and SIMD intrinsics on the CPU, and GPU kernels parallelized over both the dimensionality of the ODE system and number of trajectories. On the GPU, we draw particular attention to the problem of variable task-length among threads of the same warp, proposing a lightweight strategy of assigning multiple data items to each thread to reduce the prevalence of redundant operations. A simple analysis suggests that the strategy can draw performance close to that of ideal parallelism, while empirical results demonstrate up to a 10 percent improvement over the standard approach.