AVX-512 Instruction Research Articles

Computing the sparse matrix vector product using block-based kernels without zero padding on processors with AVX-512 instructions.

The sparse matrix-vector product (SpMV) is a fundamental operation in many scientific applications from various fields. The High Performance Computing (HPC) community has therefore continuously invested a lot of effort to provide an efficient SpMV kernel on modern CPU architectures. Although it has been shown that block-based kernels help to achieve high performance, they are difficult to use in practice because of the zero padding they require. In the current paper, we propose new kernels using the AVX-512 instruction set, which makes it possible to use a blocking scheme without any zero padding in the matrix memory storage. We describe mask-based sparse matrix formats and their corresponding SpMV kernels highly optimized in assembly language. Considering that the optimal blocking size depends on the matrix, we also provide a method to predict the best kernel to be used utilizing a simple interpolation of results from previous executions. We compare the performance of our approach to that of the Intel MKL CSR kernel and the CSR5 open-source package on a set of standard benchmark matrices. We show that we can achieve significant improvements in many cases, both for sequential and for parallel executions. Finally, we provide the corresponding code in an open source library, called SPC5.

Optimizing polynomial convolution for NTRUEncrypt

$\sf{ NTRUEncrypt}$ is one of the most promising candidates for quantum-safe cryptography. In this paper, we focus on the $\sf{ NTRU743}$ parameter set. We give a report on all known attacks against this parameter set and show that it delivers 256 bits of security against classical attackers and 128 bits of security against quantum attackers. We then present a parameter-dependent optimization using a tailored hierarchy of multiplication algorithms as well as the Intel AVX2 instructions, and show that this optimization is constant-time. Our implementation is two to three times faster than the reference implementation of $\sf{ NTRUEncrypt}$ .

DITVA: Dynamic Inter-Thread Vectorization Architecture

In the Single-Program Multiple-Data (SPMD) programming model, threads of an application exhibit very similar control flows and often execute the same instructions, but on different data. In this paper, we propose the Dynamic Inter-thread Vectorization Architecture (DITVA) to leverage the implicit Data Level Parallelism that exists across threads on SPMD applications.By assembling dynamic vector instructions at runtime, DITVA extends an in-order SMT processor with a dynamic inter-thread vector execution mode akin to the Single-Instruction, Multiple-Thread model of Graphics Processing Units. In this mode, multiple scalar threads running in lockstep share a single instruction stream and their respective instruction instances are aggregated into SIMD instructions. DITVA can leverage existing SIMD units and maintains binary compatibility with existing CPU architectures. To balance thread- and data-level parallelism, threads are statically grouped into fixed-size independently scheduled warps. Additionally, to maximize dynamic vectorization opportunities, we adapt the fetch steering policy to favor thread synchronization within warps and thus improve lockstep execution.Our experimental evaluation of the DITVA architecture on the SPMD applications from the PARSEC and Rodinia OpenMP benchmarks show that a 4-warp × 4-lane 4-issue DITVA architecture with a realistic bank-interleaved cache achieves 1.55× higher performance compared to a 4-thread 4-issue SMT architecture with AVX instructions, while fetching and issuing 51% fewer instructions, and achieving an overall 24% energy reduction. DITVA also enables applications limited by memory to scale with higher bandwidth architectures. For instance, when the bandwidth is increased from 2GB/s to 16GB/s, we find that memory bound applications show an improvement in performance by 3× in comparison with the baseline SMT. Therefore, DITVA appears as a cost-effective design for achieving very high single-core performance on SPMD parallel sections.

Correction to: CPU-based real-time maximim intensity projection via fast matrix transposition using parallelization operations with AVX instruction set

Rapid visualization is essential for maximum intensity projection (MIP) rendering, since the acquisition of a perceptual depth can require frequent changes of a viewing direction. In this paper, we propose a CPU-based real-time MIP method that uses parallelization operations with the AVX instruction set. We improve shear-warp based MIP rendering by resolving the bottle-neck problems of the previous method of a matrix transposition. We propose a novel matrix transposition method using the AVX instruction set to minimize bottle-neck problems. Experimental results show that the speed of MIP rendering on general CPU is faster than 20 frame-per-second (fps) for a 512 × 512 × 552 volume dataset. Our matrix transposition method can be applied to other image processing algorithms for faster processing.

CPU-based real-time maximim intensity projection via fast matrix transposition using parallelization operations with AVX instruction set

CPU-based real-time maximim intensity projection via fast matrix transposition using parallelization operations with AVX instruction set

Technology beats algorithms (in exact string matching)

SummaryMore than 120 algorithms have been developed for exact string matching within the last 40 years. We show by experiments that the naïve algorithm exploiting SIMD instructions of modern CPUs (with symbols compared in a special order) is the fastest one for patterns of length up to about 50 symbols and extremely good for longer patterns and small alphabets. The algorithm compares 16 or 32 characters in parallel by applying SSE2 or AVX2 instructions, respectively. Moreover, it uses loop peeling to further speed up the searching phase. We tried several orders for comparisons of pattern symbols, and the increasing order of their probabilities in the text was the best.

Exploiting vector instructions with generalized stream fusion

Ideally, a program written as a composition of concise, self-contained components should perform as well as the equivalent hand-written version where the functionality of what was many components has been manually combined into a monolithic implementation. That is, programmers should not have to sacrifice code clarity or good software engineering practices to obtain performance---we want compositionality without a performance penalty. This work shows how to attain this goal for high-level Haskell in the domain of sequence-processing functions, which includes applications such as array processing. Prior work on stream fusion shows how to automatically transform some high-level sequence-processing functions into efficient implementations. It has been used to great effect in Haskell libraries for manipulating byte arrays, Unicode text, and unboxed vectors. However some operations, like vector append, do not perform well within the stream fusion framework. Others, like SIMD computation using the SSE and AVX instructions available on modern x86 chips, do not seem to fit in the stream fusion framework at all. We describe generalized stream fusion, which solves these issues through a careful choice of stream representation. Benchmarks show that high-level Haskell code written using our compiler and libraries can produce code that is faster than both compiler- and hand-vectorized C.

Automated Compiler Optimization of Multiple Vector Loads/Stores

With widening vectors and the proliferation of advanced vector instructions in today’s processors, vectorization plays an ever-increasing role in delivering application performance. Achieving the performance potential of this vector hardware has required significant support from the software level such as new explicit vector programming models and advanced vectorizing compilers. Today, with the combination of these software tools plus new SIMD ISA extensions like gather/scatter instructions it is not uncommon to find that even codes with complex and irregular data access patterns can be vectorized. In this paper we focus on these vectorized codes with irregular accesses, and show that while the best-in-class Intel Compiler Vectorizer does indeed provide speedup through efficient vectorization, there are some opportunities where clever program transformations can increase performance further. After identifying these opportunities, this paper describes two automatic compiler optimizations to target these data access patterns. The first optimization focuses on improving the performance for a group of adjacent gathers/scatters. The second optimization improves performance for a group of stencil vector accesses using more efficient SIMD instructions. Both optimizations are now implemented in the 17.0 version of the Intel Compiler. We evaluate the optimizations using an extensive set of micro-kernels, representative benchmarks and application kernels. On these benchmarks, we demonstrate performance gains of 3–750% on the Intel $${\textregistered }$$ Xeon processor (Haswell—HSW), up to 25% on the Intel $${\textregistered }$$ Xeon $$\hbox {Phi}^{\mathrm{TM}}$$ coprocessor (Knights Corner—KNC), and up to 430% on the Intel $${\textregistered }$$ Xeon $$\hbox {Phi}^{\mathrm{TM}}$$ processor with AVX-512 instructions support (Knights Landing—KNL).

High Performance of Hash-based Signature Schemes

Hash-based signature schemes, whose security is based on properties of the underlying hash functions, are promising candidates to be quantum-safe digital signatures schemes. In this work, we present a software implementation of two recent standard proposals for hash-based signature schemes, Leighton and Micali Signature (LMS) scheme and Extended Merkle Signature Scheme (XMSS), using a set of AVX2 instructions on Intel processors. The implementation uses several optimization techniques for speeding up the underlying hash functions SHA2 or SHA3, and other building block functions which lead to high performance for signature operations on both schemes. On an Intel Skylake processor, using a tree of height 60 with 12 layers, the signing operation for XMSS takes 3,841,199 cycles (1,043 signatures per second) at 128-bit security level (against quantum attacks). For an equivalent security, the LMS system computes a signature in 1,307,376 cycles (3,065 signatures per second). We also provide the first comparative performance results for signing and verification of both schemes using different parameters. The results of our implementation indicate that both schemes LMS and XMSS can achieve high performance using vector instructions on modern processors.

An efficient and portable SIMD algorithm for charge/current deposition in Particle-In-Cell codes

In current computer architectures, data movement (from die to network) is by far the most energy consuming part of an algorithm (≈20pJ/word on-die to ≈10,000 pJ/word on the network). To increase memory locality at the hardware level and reduce energy consumption related to data movement, future exascale computers tend to use many-core processors on each compute nodes that will have a reduced clock speed to allow for efficient cooling. To compensate for frequency decrease, machine vendors are making use of long SIMD instruction registers that are able to process multiple data with one arithmetic operator in one clock cycle. SIMD register length is expected to double every four years. As a consequence, Particle-In-Cell (PIC) codes will have to achieve good vectorization to fully take advantage of these upcoming architectures. In this paper, we present a new algorithm that allows for efficient and portable SIMD vectorization of current/charge deposition routines that are, along with the field gathering routines, among the most time consuming parts of the PIC algorithm. Our new algorithm uses a particular data structure that takes into account memory alignment constraints and avoids gather/scatter instructions that can significantly affect vectorization performances on current CPUs. The new algorithm was successfully implemented in the 3D skeleton PIC code PICSAR and tested on Haswell Xeon processors (AVX2-256 bits wide data registers). Results show a factor of ×2 to ×2.5 speed-up in double precision for particle shape factor of orders 1–3. The new algorithm can be applied as is on future KNL (Knights Landing) architectures that will include AVX-512 instruction sets with 512 bits register lengths (8 doubles/16 singles). Program summaryProgram Title: vec_depositionProgram Files doi:http://dx.doi.org/10.17632/nh77fv9k8c.1Licensing provisions: BSD 3-ClauseProgramming language: Fortran 90External routines/libraries: OpenMP>4.0Nature of problem: Exascale architectures will have many-core processors per node with long vector data registers capable of performing one single instruction on multiple data during one clock cycle. Data register lengths are expected to double every four years and this pushes for new portable solutions for efficiently vectorizing Particle-In-Cell codes on these future many-core architectures. One of the main hotspot routines of the PIC algorithm is the current/charge deposition for which there is no efficient and portable vector algorithm.Solution method: Here we provide an efficient and portable vector algorithm of current/charge deposition routines that uses a new data structure, which significantly reduces gather/scatter operations. Vectorization is controlled using OpenMP 4.0 compiler directives for vectorization which ensures portability across different architectures.Restrictions: Here we do not provide the full PIC algorithm with an executable but only vector routines for current/charge deposition. These scalar/vector routines can be used as library routines in your 3D Particle-In-Cell code. However, to get the best performances out of vector routines you have to satisfy the two following requirements: (1) Your code should implement particle tiling (as explained in the manuscript) to allow for maximized cache reuse and reduce memory accesses that can hinder vector performances. The routines can be used directly on each particle tile. (2) You should compile your code with a Fortran 90 compiler (e.g Intel, gnu or cray) and provide proper alignment flags and compiler alignment directives (more details in README file).

Horizontal vectorization of electron repulsion integrals.

We present an efficient implementation of the Obara-Saika algorithm for the computation of electron repulsion integrals that utilizes vector intrinsics to calculate several primitive integrals concurrently in a SIMD vector. Initial benchmarks display a 2-4 times speedup with AVX instructions over comparable scalar code, depending on the basis set. Speedup over scalar code is found to be sensitive to the level of contraction of the basis set, and is best for (lAlB|lClD) quartets when lD = 0 or lB=lD=0, which makes such a vectorization scheme particularly suitable for density fitting. The basic Obara-Saika algorithm, how it is vectorized, and the performance bottlenecks are analyzed and discussed. © 2016 Wiley Periodicals, Inc.

Algorithm 967

The solution of a constant-coefficient elliptic Partial Differential Equation (PDE) can be computed using an integral transform: A convolution with the fundamental solution of the PDE, also known as a volume potential. We present a Fast Multipole Method (FMM) for computing volume potentials and use them to construct spatially adaptive solvers for the Poisson, Stokes, and low-frequency Helmholtz problems. Conventional N-body methods apply to discrete particle interactions. With volume potentials, one replaces the sums with volume integrals. Particle N-body methods can be used to accelerate such integrals. but it is more efficient to develop a special FMM. In this article, we discuss the efficient implementation of such an FMM. We use high-order piecewise Chebyshev polynomials and an octree data structure to represent the input and output fields and enable spectrally accurate approximation of the near-field and the Kernel Independent FMM (KIFMM) for the far-field approximation. For distributed-memory parallelism, we use space-filling curves, locally essential trees, and a hypercube-like communication scheme developed previously in our group. We present new near and far interaction traversals that optimize cache usage. Also, unlike particle N-body codes, we need a 2:1 balanced tree to allow for precomputations. We present a fast scheme for 2:1 balancing. Finally, we use vectorization, including the AVX instruction set on the Intel Sandy Bridge architecture to get better than 50% of peak floating-point performance. We use task parallelism to employ the Xeon Phi on the Stampede platform at the Texas Advanced Computing Center (TACC). We achieve about 600 gflop /s of double-precision performance on a single node. Our largest run on Stampede took 3.5s on 16K cores for a problem with 18 e +9 unknowns for a highly nonuniform particle distribution (corresponding to an effective resolution exceeding 3 e +23 unknowns since we used 23 levels in our octree).

New version of hex-ecs, the B-spline implementation of exterior complex scaling method for solution of electron–hydrogen scattering

We provide an updated version of the program hex-ecs originally presented in Comput. Phys. Commun. 185 (2014) 2903–2912. The original version used an iterative method preconditioned by the incomplete LU factorization (ILU), which–though very stable and predictable–requires a large amount of working memory. In the new version we implemented a “separated electrons” (or “Kronecker product approximation”, KPA) preconditioner as suggested by Bar-On et al., Appl. Num. Math. 33 (2000) 95–104. This preconditioner has much lower memory requirements, though in return it requires more iterations to reach converged results. By careful choice between ILU and KPA preconditioners one is able to extend the computational feasibility to larger calculations.Secondly, we added the option to run the KPA preconditioner on an OpenCL device (e.g. GPU). GPUs have generally better memory access times, which speeds up particularly the sparse matrix multiplication. New version program summaryProgram title: hex-ecsCatalogue identifier: AETI_v2_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AETI_v2_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: MIT LicenseNo. of lines in distributed program, including test data, etc.: 73693No. of bytes in distributed program, including test data, etc.: 520475Distribution format: tar.gzProgramming language: C++11.Computer: Any recent CPU, preferably 64-bit. Computationally intensive parts can be run on GPU (tested on AMD Tahiti and NVidia TitanX models).Operating system: Tested on Windows 10 and various Linux distributions.RAM: Depends on the problem solved and particular setup; KPA test run uses apx. 300 MiB.Classification: 2.4.Catalogue identifier of previous version: AETI_v2_0Journal reference of previous version: Comput. Phys. Comm. 185 (2014) 2903External routines: GSL [1], UMFPACK [2], BLAS and LAPACK (ideally threaded OpenBLAS [3]).Does the new version supersede the previous version?: YesNature of problem: Solution of the two-particle Schrödinger equation in central field.Solution method: The two-electron states are expanded into angular momentum eigenstates, which gives rise to the coupled bi-radial equations. The bi-radially dependent solution is then represented in a B-spline product basis, which transforms the set of equations into a large matrix equation in this basis. The boundary condition is of Dirichlet type, thanks to the use of the exterior complex scaling method, which extends the coordinates into the complex plane. The matrix equation is then solved by preconditioned conjugated orthogonal conjugate gradient method (PCOCG) [4].Reasons for new version: The original program has been updated to achieve better performance. Also, some external dependencies have been removed (HDF5, FFTW3), which simplifies deployment.Summary of revisions: We implemented a new preconditioner introduced in [5], both for general CPU and also for an arbitrary OpenCL device (e.g. GPU) conforming to the OpenCL 2.0 specification. Furthermore, many other minor improvements have been made, particularly with the intention of reducing the memory requirements. With appropriate switches the program now does not precompute the used matrices and only calculates their elements on the fly. This is aided also by the vectorized B-spline evaluation function, which can now make use of AVX instructions when a single B-spline is being evaluated at several points. The accompanying tools hex-db and hex-dwba [6] have been also updated to use the shared code base.Running time: KPA test run — apx. 2 minutes on Intel i7-4790K (4 threads)

Optimization of GEMV on Intel AVX Processor

To improve the performance of BLAS 2 GEMV subroutine under the latest instruction set, Intel AVX, this paper presents a new approach to analyze the new generation instruction set and enhance the efficiency of current data-oriented math subroutines. The whole optimizing process involves memory access optimization, SIMD optimization and parallel optimization. Also, this paper shows the comparison between the traditional SSE instruction set and the AVX instruction set. Experiments show that the optimized GEMV function has obtained considerable increase on performance. Compared with the Intel MKL, GotoBLAS, ATLAS, this optimized GEMV exceeds these BLAS implementations from 5% to 10%.

RNGAVXLIB: Program library for random number generation, AVX realization

Abstract The library RNGSSELIB for random number generators (RNGs) based upon the SSE2 command set is presented. The library contains realization of a number of modern and most reliable generators. Usage of SSE2 command set allows to substantially improve performance of the generators. Three new RNG realizations are also constructed. We present detailed analysis of the speed depending on compiler usage and associated optimization level, as well as results of extensive statistical testing for all generators using available test packages. Fast SSE implementations produce exactly the same output sequence as the original algorithms. Program summary Program title: RNGSSELIB Catalogue identifier: AEIT_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEIT_v1_0.html Program obtainable from: CPC Program Library, Queenʼs University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 4177 No. of bytes in distributed program, including test data, etc.: 21 228 Distribution format: tar.gz Programming language: C. Computer: PC. Operating system: UNIX, Windows. RAM: 1 Mbytes Classification: 4.13. Nature of problem: Any calculation requiring uniform pseudorandom number generator, in particular, Monte Carlo calculations. Solution method: The library contains realization of a number of modern and reliable generators: mt19937 , mrg32k3a and lfsr113 . Also new realizations for the method based on parallel evolution of an ensemble of dynamical systems are constructed: GM19 , GM31 and GM61 . The library contains both usual realizations and realizations based on SSE command set. Usage of SSE commands allows the performance of all generators to be substantially improved. Restrictions: For SSE realizations of the generators, Intel or AMD CPU supporting SSE2 command set is required. In order to use the realization lfsr113sse , CPU must support SSE4 command set. Running time: Running time is of the order of 20 sec for generating 109 pseudorandom numbers with a PC based on Intel Core i7-940 CPU. Running time is analysed in detail in Section 5 of the paper.

Exploiting hardware capabilities in interior point methods

The increase of computer performance continues to support the practice of large-scale optimization. Computers with multiple computing cores and vector processing capabilities are now widely available. We investigate how the recently introduced Advanced Vector Instruction (AVX) set on Intel-compatible architectures can be exploited in interior point methods for linear and nonlinear optimization. We focus on data structures and implementation techniques that utilize the new vector instructions. Our numerical experiments demonstrate that the AVX instruction set provides a significant performance boost in our implementation on large-scale problem that have significant fill-in in the sparse Cholesky factorization, achieving up to 100 gigaflops performance on a standard desktop computer on linear optimization problems for which the required Cholesky factorization is relatively dense.

Parallel direct solver for solving systems of linear equations resulting from finite element method on multi-core desktops and workstations

This paper presents the PARFES—parallel sparse direct solver for solving systems of linear equations resulting from the finite element method applied to problems of structural and solid mechanics. PARFES is intended for use on shared-memory multi-core desktops, laptops and workstations. The solver’s high performance relies on the use of a specific algorithm for the procedure of block matrix multiplication using AVX instructions for the vectorization of computations, and the parallelizing approach that ensures a balanced processor load. The block looking-left factorization algorithm allows for the use of disk memory if the dimensionality of problem exceeds the capacity of RAM.

An energy‐aware performance analysis of SWIMM: Smith–Waterman implementation on Intel's Multicore and Manycore architectures

SummaryAlignment is essential in many areas such as biological, chemical and criminal forensics. The well‐known Smith–Waterman (SW) algorithm is able to retrieve the optimal local alignment with quadratic time and space complexity. There are several implementations that take advantage of computing parallelization, such as manycores, FPGAs or GPUs, in order to reduce the alignment effort. In this research, we adapt, develop and tune the SW algorithm named SWIMM on a heterogeneous platform based on Intel's Xeon and Xeon Phi coprocessor. SWIMM is a free tool available in a public git repository https://github.com/enzorucci/SWIMM. We efficiently exploit data and thread‐level parallelism, reaching up to 380 GCUPS on heterogeneous architecture, 350 GCUPS for the isolated Xeon and 50 GCUPS on Xeon Phi. Despite the heterogeneous implementation obtaining the best performance, it is also the most energy‐demanding. In fact, we also present a trade‐off analysis between performance and power consumption. The greenest configuration is based on an isolated multicore system that exploits AVX2 instruction set architecture reaching 1.5 GCUPS/Watts. Copyright © 2015 John Wiley & Sons, Ltd.

Embedded GPU and multicore processors for emotional-based mobile robotic agents

Control architectures based on emotions are becoming promising solutions for the implementation of future robotic systems. The basic controllers of this architecture are the emotional processes that decide which behaviors the robot must activate to fulfill the objectives. The number of emotional processes increases (hundreds of millions/s) with the complexity level of the application, limiting the processing capacity of a main processor to solve the complex problems. Fortunately, the potential parallelism of emotional processes permits their execution in parallel, hence enabling the computing power to tackle the complex dynamic problems. In this paper, Graphic Processing Unit (GPU), multicore processors and single instruction multiple data (SIMD) instructions are used to provide parallelism for the emotional processes. Different GPUs, multicore processors and SIMD instruction sets are evaluated and compared to analyze their suitability to cope with robotic applications. The applications are set-up taking into account different environmental conditions, robot dynamics and emotional states. Experimental results show that, despite the fact that GPUs have a bottleneck in the data transmission between the host and the device, the evaluated GTX 670 GPU provides a performance of more than one order of magnitude greater than the initial implementation of the architecture on a single core. Thus, all complex proposed application problems can be solved using the GPU technology in contrast to the first prototype where only 55% of them could be solved. Using AVX SIMD instructions, the performance of the architecture is increased in 3.25 times in relation to the first implementation. Thus, from the 27 proposed applications about 88.8% are solved. In the case of the SSE SIMD instructions, the performance is almost doubled and the robot could solve about 74% of the proposed application problems. The use of AVX and SSE SIMD instructions provides almost the same performance as a quad- and a dual-core, respectively, with the advantage that they do not add any additional hardware cost.

Exploiting vector instructions with generalized stream fusion

Stream fusion is a powerful technique for automatically transforming high-level sequence-processing functions into efficient implementations. It has been used to great effect in Haskell libraries for manipulating byte arrays, Unicode text, and unboxed vectors. However, some operations, like vector append, still do not perform well within the standard stream fusion framework. Others, like SIMD computation using the SSE and AVX instructions available on modern x86 chips, do not seem to fit in the framework at all. In this paper we introduce generalized stream fusion, which solves these issues. The key insight is to bundle together multiple stream representations, each tuned for a particular class of stream consumer. We also describe a stream representation suited for efficient computation with SSE instructions. Our ideas are implemented in modified versions of the GHC compiler and vector library. Benchmarks show that high-level Haskell code written using our compiler and libraries can produce code that is faster than both compiler- and hand-vectorized C.