Single Instruction Multiple Data Units Research Articles

Micro-kernels for portable and efficient matrix multiplication in deep learning

We provide a practical demonstration that it is possible to systematically generate a variety of high-performance micro-kernels for the general matrix multiplication (gemm) via generic templates which can be easily customized to different processor architectures and micro-kernel dimensions. These generic templates employ vector intrinsics to exploit the SIMD (single instruction, multiple data) units in current general-purpose processors and, for the particular type of gemm problems encountered in deep learning, deliver a floating-point throughput rate on par with or even higher than that obtained with conventional, carefully tuned implementations of gemm in current linear algebra libraries (e.g., BLIS, AMD AOCL, ARMPL). Our work exposes the structure of the template-based micro-kernels for ARM Neon (128-bit SIMD), ARM SVE (variable-length SIMD) and Intel AVX512 (512-bit SIMD), showing considerable performance for an NVIDIA Carmel processor (ARM Neon), a Fujitsu A64FX processor (ARM SVE) and on an AMD EPYC 7282 processor (256-bit SIMD).

Auto-tuning for floating-point precision with Discrete Stochastic Arithmetic

The type length chosen for floating-point numbers (e.g. 32 bits or 64 bits) may have an impact on the execution time, especially on SIMD (Single Instruction Multiple Data) units. Furthermore optimizing the types used in a numerical simulation causes a reduction of the data volume that is possibly transferred. In this paper we present PROMISE, a tool that makes it possible to optimize the numerical types in a program by taking into account the requested accuracy on the computed results. With PROMISE the numerical quality of results is verified using DSA (Discrete Stochastic Arithmetic) that enables one to estimate round-off errors. The search for a suitable type configuration is performed with a reasonable complexity thanks to the delta debugging algorithm. The PROMISE tool has been successfully tested on programs implementing several numerical algorithms including linear system solving and also on an industrial code that solves the neutron transport equations.

PVL: Parallelization and Vectorization of Affine Perfectly Nested-Loops Considering Data Locality on Short-Vector Multicore Processors using Intrinsic Vectorization

There is an urgent need for high-performance computations. Cores and Single Instruction Multiple Data (SIMD) units are important resources of modern architectures to speed up the execution of programs. Also, the importance of the data locality cannot be neglected in computations. Using cores, SIMD units, and data locality simultaneously is critical to gain peak performance of the architecture. But, there are a few research efforts trying to consider these three resources at the same time. There is a challenge in choosing loops which could be whether run on SIMD units or cores for vectorization and parallelization, respectively. This paper proposes an approach, named PVL, for parallelization and vectorization of nested loops considering data locality based on the polyhedral model on short-vector multicore processors. More precisely, PVL, tries to satisfy dependences in the middle levels of nested loop (the levels between outermost and innermost levels) while trying to move dependence-free loops to the outermost and innermost position in order to parallelize and vectorize them, respectively. The experimental results show that the proposed approach, PVL, is significantly effective compared to the other approaches.

CPU-GPU2Trigeneous Computing for Iterative Reconstruction in Computed Tomography

In this paper, we present methods to efficiently parallelize iterative 3D image reconstruction by exploiting trigeneous devices (three different types of device) at the same time: a CPU, an integrated GPU, and a discrete GPU. We first present a technique that exploits single instruction multiple data (SIMD) architectures in GPUs. Then, we propose a performance estimation model, based on which we can easily find the optimal data partitioning on trigeneous devices. We found that the performance significantly varies by up to 6.23 times, depending on how SIMD units in GPUs are accessed. Then, by using trigeneous devices and the proposed estimation models, we achieve optimal partitioning and throughput, which corresponds to a 9.4% further improvement, compared to discrete GPU-only execution.

Coherent Photon Mapping on the Intel MIC Architecture

Photon mapping is a global illumination algorithm which is composed of two steps: photon tracing and photon searching. During photon searching step, each shading point needs to search the photon-tree to find k-neighbouring photons for reflected radiance estimation. As the number of shading points and the size of photon-tree are dramatically large, the photon searching step is time consuming. We propose a parallel photon searching algorithm by using radiance estimation approach for coherent shading points on the Intel® Many Integrated Core (MIC) Architecture. In order to efficiently use single instruction multiple data (SIMD) units, shading points are clustered by similarity first (every cluster contains 16 shading-points), and an initial neighbouring scope is searched from the photon-tree for each cluster. Then we use 16-wide SIMD units by performing k-NN searching from the initial neighbouring scope for those 16 shading-points in a cluster in parallel. We use the method to simulate some global illumination scenes on Intel® Xeon® processors and Intel® Xeon® Phi™ coprocessors. The comparison results with existing photon mapping techniques indicate that our method gives significant improvement in speed with the same accuracy.

A Matrix–Matrix Multiplication methodology for single/multi-core architectures using SIMD

In this paper, a new methodology for speeding up Matrix—Matrix Multiplication using Single Instruction Multiple Data unit, at one and more cores having a shared cache, is presented. This methodology achieves higher execution speed than ATLAS state of the art library (speedup from 1.08 up to 3.5), by decreasing the number of instructions (load/store and arithmetic) and the data cache accesses and misses in the memory hierarchy. This is achieved by fully exploiting the software characteristics (e.g. data reuse) and hardware parameters (e.g. data caches sizes and associativities) as one problem and not separately, giving high quality solutions and a smaller search space.

A flexible algorithm for calculating pair interactions on SIMD architectures

Calculating interactions or correlations between pairs of particles is typically the most time-consuming task in particle simulation or correlation analysis. Straightforward implementations using a double loop over particle pairs have traditionally worked well, especially since compilers usually do a good job of unrolling the inner loop. In order to reach high performance on modern CPU and accelerator architectures, single-instruction multiple-data (SIMD) parallelization has become essential. Avoiding memory bottlenecks is also increasingly important and requires reducing the ratio of memory to arithmetic operations. Moreover, when pairs only interact within a certain cut-off distance, good SIMD utilization can only be achieved by reordering input and output data, which quickly becomes a limiting factor. Here we present an algorithm for SIMD parallelization based on grouping a fixed number of particles, e.g. 2, 4, or 8, into spatial clusters. Calculating all interactions between particles in a pair of such clusters improves data reuse compared to the traditional scheme and results in a more efficient SIMD parallelization. Adjusting the cluster size allows the algorithm to map to SIMD units of various widths. This flexibility not only enables fast and efficient implementation on current CPUs and accelerator architectures like GPUs or Intel MIC, but it also makes the algorithm future-proof. We present the algorithm with an application to molecular dynamics simulations, where we can also make use of the effective buffering the method introduces.

Modeling, validation, and co-design of IBM Blue Gene/Q: Tools and examples

Major architectural innovations in the compute node have been introduced in the IBM Blue Gene®/Q, including programmable Level 1 (L1) cache data prefetching units to hide memory access latency, hardware support for transactional memory (TM) and speculative execution (SE), an enhanced five-dimensional integrated torus network, and a high-performance quad floating-point SIMD (single-instruction, multiple-data) unit. In this paper, we present the tools and methodology that we used to model, co-design, and validate these new features from early concept phase through design implementation. Early in the design cycle, we made extensive use of an architectural simulator, BGQSim, capable of executing unmodified binary Blue Gene/Q code for single as well as multiple nodes. As the hardware description language for the chip implementation became available, we complemented BGQSim with a cycle-accurate and cycle-reproducible, large-scale field-programmable gate array-based platform, Twinstar, to validate the implementation against performance targets and functional specifications. Through specific examples, we show the effectiveness of these tools in co-developing the hardware and software of Blue Gene/Q, allowing us to meet the design targets at an aggressive project schedule.

Determination of performance characteristics of scientific applications on IBM Blue Gene/Q

The IBM Blue Gene®/Q platform presents scientists and engineers with a rich set of features such as 16 cores per chip sharing a Level 2 cache, a wide SIMD (single-instruction, multiple-data) unit, a five-dimensional torus network, and support for collective operations. An especially important feature is that the cores have four hardware threads, which makes it possible to hide latencies and obtain a high fraction of the peak issue rate from each core. All of these resources present unique performance-tuning opportunities on Blue Gene/Q. We provide an overview of several important applications and solvers and study them on Blue Gene/Q using performance counters and Message Passing Interface profiles. We discuss how Blue Gene/Q tools help us understand the interaction of the application with the and software layers and provide guidance for optimization. On the basis of our analysis, we discuss code improvement strategies targeting Blue Gene/Q. Information about how these algorithms map to the Blue Gene® architecture is expected to have an impact on future system design as we move to the exascale era.

DMA++: On the Fly Data Realignment for On-Chip Memories

Multimedia extensions based on Single-Instruction Multiple-Data (SIMD) units are widespread. They have been used, for some time, in processors and accelerators (e.g., the Cell SPEs). SIMD units usually have significant memory alignment constraints in order to meet power requirements and design simplicity. This increases the complexity of the code generated by the compiler as, in the general case, the compiler cannot be sure of the proper alignment of data. For that, the ISA provides either unaligned memory load and store instructions, or a special set of instructions to perform realignments in software. In this paper, we propose a hardware realignment unit that takes advantage of the DMA transfers needed in accelerators with local memories. While the data are being transferred, it is realigned on the fly by our realignment unit, and stored at the desired alignment in the accelerator memory. This mechanism can help programmers to better organize data in the accelerator memory so that the accelerator can possibly access the data with no special instructions. Finally, the data are realigned properly also when put back to main memory. Our experiments with nine applications show that with our approach, the bandwidth of the DMA transfers is not penalized.

Understanding sources of ineffciency in general-purpose chips

Scaling the performance of a power limited processor requires decreasing the energy expended per instruction executed, since energy/op * op/second is power. To better understand what improvement in processor efficiency is possible, and what must be done to capture it, we quantify the sources of the performance and energy overheads of a 720p HD H.264 encoder running on a general-purpose four-processor CMP system. The initial overheads are large: the CMP was 500 x less energy efficient than an Application Specific Integrated Circuit (ASIC) doing the same job. We explore methods to eliminate these overheads by transforming the CPU into a specialized system for H.264 encoding. Broadly applicable optimizations like single instruction, multiple data (SIMD) units improve CMP performance by 14 x and energy by 10x, which is still 50x worse than an ASIC. The problem is that the basic operation costs in H.264 are so small that even with a SIMD unit doing over 10 ops per cycle, 90% of the energy is still overhead. Achieving ASIC-like performance and effciency requires algorithm-specifc optimizations. For each subalgorithm of H.264, we create a large, specialized functional/storage unit capable of executing hundreds of operations per instruction. This improves energy effciency by 160x (instead of 10x), and the final customized CMP reaches the same performance and within 3x of an ASIC solution's energy in comparable area.

Returning Control to the Programmer

Server and workstation hardware architecture is continually improving, yet interpreted languages have failed to keep pace with the proper utilization of modern processors. SIMD (single instruction, multiple data) units are available in nearly every current desktop and server processor and are greatly underutilized, especially with interpreted languages. If multicore processors continue their current growth pattern, interpreted-language performance will begin to fall behind, since current native compilers and languages offer better automated SIMD optimization and direct SIMD mapping support. As each core in commercial x86 multicore processors includes a dedicated SIMD unit, the performance disparity will grow exponentially as long as the available SIMD units remain underutilized in interpreted-language environments.