SIMD Research Articles

Survey-scale discovery-based research processes: Evaluating a bespoke visualisation environment for astronomical survey data

Abstract Next-generation astronomical surveys naturally pose challenges for human-centred visualisation and analysis workflows that currently rely on the use of standard desktop display environments. While a significant fraction of the data preparation and analysis will be taken care of by automated pipelines, crucial steps of knowledge discovery can still only be achieved through various level of human interpretation. As the number of sources in a survey grows, there is need to both modify and simplify repetitive visualisation processes that need to be completed for each source. As tasks such as per-source quality control, candidate rejection, and morphological classification all share a single instruction, multiple data (SIMD) work pattern, they are amenable to a parallel solution. Selecting extragalactic neutral hydrogen (Hi) surveys as a representative example, we use system performance benchmarking and the visual data and reasoning methodology from the field of information visualisation to evaluate a bespoke comparative visualisation environment: the encube visual analytics framework deployed on the 83 Megapixel Swinburne Discovery Wall. Through benchmarking using spectral cube data from existing Hi surveys, we are able to perform interactive comparative visualisation via texture-based volume rendering of 180 three-dimensional (3D) data cubes at a time. The time to load a configuration of spectral cubes scale linearly with the number of voxels, with independent samples of 180 cubes (8.4 Gigavoxels or 34 Gigabytes) each loading in under 5 min. We show that parallel comparative inspection is a productive and time-saving technique which can reduce the time taken to complete SIMD-style visual tasks currently performed at the desktop by at least two orders of magnitude, potentially rendering some labour-intensive desktop-based workflows obsolete.

AVX512Crypto: Parallel Implementations of Korean Block Ciphers Using AVX-512

Cryptographic algorithms are widely used as the foundation of various security systems and applications (e.g., secure communication, blockchain systems, and cloud services). A block cipher is an essential cryptographic algorithm to achieve confidentiality. This paper proposes parallel implementations of Korean block ciphers using Advanced Vector eXtension (AVX)-512, which is a new Single instruction, multiple data (SIMD) instruction set that has recently been integrated into several high-end Intel central processing unit (CPU). Target algorithms are LEA, HIGHT, and CHAM block ciphers. Additionally, this paper also proposes applicable implementing techniques, which are designed for each algorithm. These enable to use of parallel processing instructions in AVX-512 properly for each algorithm. The proposed LEA-128 (192, 256 resp.)implementation demonstrates a performance improvement of 506.09% (323.31%, 386.76% resp.) compared to the reference code, and our HIGHT implementation exhibits a performance improvement of 520.88% compared to the reference code. In addition, CHAM-64/128 (128/256 resp.) implementation shows a performance improvement of 1,325.81% (833.61% resp.) compared to the reference code. In addition, we measured the performance with a 32MB long message. LEA-128 (192, 256 resp.) implementation showed an improvement of 556.32% (594.74%, 615.38% resp.) compared with the reference code. Also, HIGHT implementation showed 834.40%, and CHAM showed 1,332.40% (832.86% resp.) for CHAM-64/128 (CHAM-128/256 resp.), compared by the reference code. To the best of our knowledge, this is the first result of the study to optimize Korean cryptographic algorithms using the AVX-512 instruction set. The proposed methods can effectively be used in Addition, Rotation, and XOR (ARX)-based cryptographic algorithms, enabling efficient cryptographic algorithm processing in various environments such as hash-based signatures, service environments, gateway, and edge computing.

Portable and Efficient Implementation of CRYSTALS-Kyber Based on WebAssembly

With the rapid development of quantum computers capable of realizing Shor’s algorithm, existing public key-based algorithms face a significant security risk. Crystals-Kyber has been selected as the only key encapsulation mechanism (KEM) algorithm in the National Institute of Standards and Technology (NIST) Post-Quantum Cryptography (PQC) competition. In this study, we present a portable and efficient implementation of a Crystals-Kyber post-quantum KEM based on WebAssembly (Wasm), a recently released portable execution framework for high-performance web applications. Until now, most Kyber implementations have been developed with native programming languages such as C and Assembly. Although there are a few previous Kyber implementations based on JavaScript for portability, their performance is significantly lower than that of implementations based on native programming languages. Therefore, it is necessary to develop a portable and efficient Kyber implementation to secure web applications in the quantum computing era. Our Kyber software is based on JavaScript and Wasm to provide portability and efficiency while ensuring quantum security. Namely, the overall software is written in JavaScript, and the performance core parts (secure hash algorithm-3-based operations and polynomial multiplication) are written in Wasm. Furthermore, we parallelize the number theoretic transform (NTT)-based polynomial multiplication using single instruction multiple data (SIMD) functionality, which is available in Wasm. The three steps in the NTT-based polynomial multiplication have been parallelized with Wasm SIMD intrinsic functions. Our software outperforms the latest reference implementation of Kyber developed in JavaScript by ×4.02 (resp. ×4.32 and ×4.1), ×3.42 (resp. ×3.52 and ×3.44), and ×3.41 (resp. ×3.44 and ×3.38) in terms of key generation, encapsulation, and decapsulation on Google Chrome (resp. Firefox, and Microsoft Edge). As far as we know, this is the first software implementation of Kyber with Wasm technology in the web environment.

Micro-kernels for portable and efficient matrix multiplication in deep learning

We provide a practical demonstration that it is possible to systematically generate a variety of high-performance micro-kernels for the general matrix multiplication (gemm) via generic templates which can be easily customized to different processor architectures and micro-kernel dimensions. These generic templates employ vector intrinsics to exploit the SIMD (single instruction, multiple data) units in current general-purpose processors and, for the particular type of gemm problems encountered in deep learning, deliver a floating-point throughput rate on par with or even higher than that obtained with conventional, carefully tuned implementations of gemm in current linear algebra libraries (e.g., BLIS, AMD AOCL, ARMPL). Our work exposes the structure of the template-based micro-kernels for ARM Neon (128-bit SIMD), ARM SVE (variable-length SIMD) and Intel AVX512 (512-bit SIMD), showing considerable performance for an NVIDIA Carmel processor (ARM Neon), a Fujitsu A64FX processor (ARM SVE) and on an AMD EPYC 7282 processor (256-bit SIMD).

Partition-based SIMD Processing and its Application to Columnar Database Systems

The Single Instruction Multiple Data (SIMD) paradigm became a core principle for optimizing query processing in columnar database systems. Until now, only the LOAD/STORE instructions are considered to be efficient enough to achieve the expected speedups, while avoiding GATHER/SCATTER is considered almost imperative. However, the GATHER instruction offers a very flexible way to populate SIMD registers with data elements coming from non-consecutive memory locations. As we will discuss within this article, the GATHER instruction can achieve the same performance as the LOAD instruction, if applied properly. To enable the proper usage, we outline a novel access pattern allowing fine-grained, partition-based SIMD implementations. Then, we apply this partition-based SIMD processing to two representative examples from columnar database systems to experimentally demonstrate the applicability and efficiency of our new access pattern.

Accelerating the scheduling of the network resources of the next-generation optical data centers

Data centers (DCs) play a key role in the evolving IT applications and they rely heavily on the optical interconnects to improve their performance and scalability. Optically switched DCs most often exploit the slotted Time Division Multiplexing Access (TDMA) operation and the Wavelength Division Multiplexing (WDM) technology and rely on the effective scheduling of the TDMA frames to decide in real time the end-to-end connections that include the network links, switches and ports. This task becomes computationally intensive as the communication requests increase.The current paper builds on a greedy scheduling algorithm to introduce a parallel technique that accelerates the scheduling process and improves optical DC’s performance. The proposed technique handles efficiently the scheduler’s data structures, minimizes the communication among the scheduler’s processors and it is scalable. Moreover, this work presents the technique’s performance results for a variety of scheduling scenarios and DC sizes executed on an algorithm-specific Single Instruction Multiple Data (SIMD) accelerator architecture and on a Graphics Processing Unit (GPU). The performance of the GPU and the SIMD accelerator implemented on FPGA validate the parallel scheduler technique.

Secure word-level sorting based on fully homomorphic encryption

Fully homomorphic encryption enables to perform arbitrary computations over encrypted data opening a new avenue for the secure outsourced computation in cloud computing. Data sorting is a classical computational problem in computer science. Previously, all the secure sorting algorithms based on fully homomorphic encryption made use of the bit-level encryption, which requires that each bit of the input data is encrypted separately. However, the bit-level encryption incurs expensive computational overheads and a large ciphertext expansion. In this paper, we propose a secure word-level sorting scheme based on fully homomorphic encryption. We introduce a set of novel techniques to design an efficient word-level sorting algorithm. These include a new technique to compare two encrypted integers, a constant extraction technique to postprocess the output ciphertext of the comparison algorithm, an equality test method to check whether two ciphertexts encrypt the same integer and the single instruction multiple data technique to reduce the ciphertext sizes and improve the computational performance. The experimental results show that our word-level sorting algorithm outperforms the state-of-the-art bit-level sorting algorithm by more than 10 times.

Implementation of a real‐time stereo vision algorithm on a cost‐effective heterogeneous multicore platform

SummaryStereo vision is a major computer vision technique commonly used for robotics applications. Existing software implementations of this technique on general‐purpose processors offer low time‐to‐market compared to other platforms. However, such implementations can hardly achieve real‐time and their cost is usually relatively high. These issues can be solved by embedded multicore platforms. In this article, we present a low‐cost, improved software implementation of a stereo matching algorithm in the correlation stage that combines a sparse rank transform with a combination of sum of absolute differences 1‐D and 2‐D box filtering algorithms. A circular buffer scheme is used to optimize memory usage during the rank computation stage. The system runs on a heterogeneous multicore platform (ODROID XU4). Through the extensive use of single instruction multiple data Neon intrinsics, the system can process images with a size of pixels and a disparity range of 20 pixels at a rate of 111 frames per second. The proposed system can be used in mobile robot platforms that require low power consumption while delivering real‐time performance.

Multi-Gbps LDPC Decoder on GPU Devices

To meet the high throughput requirement of communication systems, the design of high-throughput low-density parity-check (LDPC) decoders has attracted significant attention. This paper proposes a high-throughput GPU-based LDPC decoder, aiming at the large-scale data process scenario, which optimizes the decoder from the perspectives of the decoding parallelism and data scheduling strategy, respectively. For decoding parallelism, the intra-codeword parallelism is fully exploited by combining the characteristics of the flooding-based decoding algorithm and GPU programming model, and the inter-codeword parallelism is improved using the single-instruction multiple-data (SIMD) instructions. For the data scheduling strategy, the utilization of off-chip memory is optimized to satisfy the demands of large-scale data processing. The experimental results demonstrate that the decoder achieves 10 Gbps throughput by incorporating the early termination mechanism on general-purpose GPU (GPGPU) devices and can also achieve a high-throughput and high-power-efficiency performance on low-power embedded GPU (EGPU) devices. Compared with the state-of-the-art work, the proposed decoder had a ×1.787 normalized throughput speedup at the same error correcting performance.

Privacy-Preserving Vertical Collaborative Logistic Regression without Trusted Third-Party Coordinator

Collaborative learning is an emerging distributed learning paradigm, which enables multiple parties to jointly train a shared machine learning (ML) model without causing the disclosure of the raw data of each party. As one of the fundamental collaborative learning algorithms, privacy-preserving collaborative logistic regression has recently gained attention from industry and academia, which utilizes cryptographic techniques to securely train joint logistic regression models across data from multiple parties. However, existing schemes have high communication and computational overhead, lose the ability to deal with high-dimensional sparse samples, cut down the accuracy of the model, or exist the risk of leaking private information. To overcome these issues, considering vertically distributed data, we propose a privacy-preserving vertical collaborative logistic regression ( P 2 VCLR) based on approximate homomorphic encryption (HE), which enables two parties to jointly train a shared model without a trusted third-party coordinator. Our scheme utilizes batching method in approximate HE to encrypt multiple data into a single ciphertext and enable a parallel processing through single instruction multiple data (SIMD) manner. We evaluate our scheme by using three publicly available datasets, the experimental results indicate that our scheme outperforms existing schemes in terms of training time and model performance.

An Application Specific Vector Processor for Efficient Massive MIMO Processing

This paper presents an implementation for a baseband massive multiple-input multiple-output (MIMO) application-specific instruction set processor (ASIP). The ASIP is geared with vector processing capabilities in the form of single instruction multiple data (SIMD), and furthermore exploits instruction level parallelism by employing a very large instruction word (VLIW) architecture. Additionally, a systolic array is built into the pipeline which is tuned to speed up matrix calculations. A parallel memory subsystem and stand-alone accelerators are integrated into the ASIP architecture in order to meet the processing requirement. The processor is synthesized in 22 nm FD-SOI technology running at a clock frequency of 800 MHz. The system achieves a maximum detection throughput of 0.75 Gb/s/mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> for a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$128\times 8$ </tex-math></inline-formula> massive MIMO system.

Compiling for the IBM Matrix Engine for Enterprise Workloads

The matrix-multiply assist (MMA) facility is the latest addition to IBM’s power instruction set architecture and first shipped in the recently introduced POWER10 processor. MMA is designed to accelerate matrix–matrix operations, such as matrix multiplication and convolution, using instructions that compute the outer product of vector-register operands. Outer product computations have been used for decades in linear algebra libraries to deliver high-performance implementations of matrix operations. Such libraries use conventional single-instruction–multiple-data (SIMD) instructions to emulate outer product operations. MMA in POWER10 is the first hardware with direct support for outer product operations released in the market. MMA operates with the widest diversity of data types compared to any accelerator design currently announced. Unleashing the high-performance enabled by MMA requires careful code generation. Two key considerations for optimal MMA code performance are 1) the choice of accumulation layout when maximizing the using the accumulators and 2) the selection of matrix access order. This article shows that over 92% of peak performance in POWER10 with MMA can be achieved when the code generation makes the right choices.

CONNA: Configurable Matrix Multiplication Engine for Neural Network Acceleration

Convolutional neural networks (CNNs) have demonstrated promising results in various applications such as computer vision, speech recognition, and natural language processing. One of the key computations in many CNN applications is matrix multiplication, which accounts for a significant portion of computation. Therefore, hardware accelerators to effectively speed up the computation of matrix multiplication have been proposed, and several studies have attempted to design hardware accelerators to perform better matrix multiplications in terms of both speed and power consumption. Typically, accelerators with either a two-dimensional (2D) systolic array structure or a single instruction multiple data (SIMD) architecture are effective only when the input matrix has shapes that are close to or similar to a square. However, several CNN applications require multiplications of non-squared matrices with various shapes and dimensions, and such irregular shapes lead to poor utilization efficiency of the processing elements (PEs). This study proposes a configurable engine for neural network acceleration, called CONNA, whose computation engine can conduct matrix multiplications with highly utilized computing units, regardless of the access patterns, shapes, and dimensions of the input matrices by changing the shape of matrix multiplication conducted in the physical array. To verify the functionality of the CONNA accelerator, we implemented CONNA as an SoC platform that integrates a RISC-V MCU with CONNA on Xilinx VC707 FPGA. SqueezeNet on CONNA achieved an inference performance of 100 frames per second (FPS) with 2.36 mm2 and 83.55 mW in a 65 nm process, improving efficiency by up to 34.1 times better than existing accelerators in terms of FPS, silicon area, and power consumption.

Privacy-Preserving Outsourced Logistic Regression on Encrypted Data from Homomorphic Encryption

Logistic regression is a data statistical technique, which is used to predict the probability that an event occurs. For some scenarios where the storage capabilities and computing resources of the data owner are limited, the data owner wants to train the logistic regression model on the cloud service provider, while the high sensitivity of training data requires effective privacy protection methods that enable efficient model training without exposing information about the training data to untrusted cloud service providers. Recently, several works have used cryptographic techniques to implement privacy-preserving logistic regression in such application scenarios. However, on large-scale training datasets, the existing works still have the problems of long model training time and poor model performance. To solve these problems, based on the homomorphic encryption (HE), we propose an efficient privacy-preserving outsourced logistic regression (P2OLR) on encrypted training data, which enables data owners to utilize the powerful storage and computing resources of cloud service providers for logistic regression analysis without exposing data privacy. Furthermore, the proposed scheme can pack multiple messages into one ciphertext and perform the same arithmetic evaluations on multiple plaintext slots by using the batching technique and single instruction multiple data (SIMD) mechanism in HE. On three public training datasets, the experimental results show that, compared with the existing schemes, the proposed scheme has better performance in terms of the encryption and decryption time of the data owner, the storage of encrypted training data, and the training time and accuracy of the model.

Tree sketch: An accurate and memory-efficient sketch for network-wide measurement

Traffic measurement provides essential information for bandwidth management and Quality of Service (QoS), which is an enabler of network status discovery. However, with the explosive growth of network traffic, traditional network measurement methods face challenges in terms of memory, computation, accuracy, especially in the high-speed networks. Network devices have no enough CPU or memory resources for collecting, counting and analyzing data streams, which makes measurement results inaccurate and unreliable. To address the problem, we design an accurate and memory-efficient Tree sketch. It consists of three different structures for heavy flow detection, Distributed Denial-of-Service (DDoS) attack detection, super-spreaders detection, respectively. The proposed heavy cuckoo algorithm stores the flows in categories according to their flow sizes reducing hash collisions in the sketch. Besides, Tree sketch employs the Single Instruction Multiple Data (SIMD) instructions to speed up packet processing. Experimental results show that F1 score of Tree sketch is 0.997 with 30 KB memory usage in heavy hitter detection, which is 0.019–0.137 higher than the state-of-the-art sketch algorithms. It is an efficient tool of traffic measurement in the backbone network and provides a good trade-off among accuracy, speed and memory usage.

An architecture-level analysis on deep learning models for low-impact computations

Deep neural networks (DNNs) have made significant achievements in a wide variety of domains. For the deep learning tasks, multiple excellent hardware platforms provide efficient solutions, including graphics processing units (GPUs), central processing units (CPUs), field programmable gate arrays (FPGAs), and application-specific integrated circuit (ASIC). Nonetheless, CPUs outperform other solutions including GPUs in many cases for the inference workload of DNNs with the support of various techniques, such as the high-performance libraries being the basic building blocks for DNNs. Thus, CPUs have been a preferred choice for DNN inference applications, particularly in the low-latency demand scenarios. However, the DNN inference efficiency remains a critical issue, especially when low latency is required under conditions with limited hardware resources, such as embedded systems. At the same time, the hardware features have not been fully exploited for DNNs and there is much room for improvement. To this end, this paper conducts a series of experiments to make a thorough study for the inference workload of prominent state-of-the-art DNN architectures on a single-instruction-multiple-data (SIMD) CPU platform, as well as with widely applicable scopes for multiple hardware platforms. The study goes into depth in DNNs: the CPU kernel-instruction level performance characteristics of DNNs including branches, branch prediction misses, cache misses, etc, and the underlying convolutional computing mechanism at the SIMD level; The thorough layer-wise time consumption details with potential time-cost bottlenecks; And the exhaustive dynamic activation sparsity with exact details on the redundancy of DNNs. The research provides researchers with comprehensive and insightful details, as well as crucial target areas for optimising and improving the efficiency of DNNs at both the hardware and software levels.

Vectorized Formulation of Newton-Euler Dynamics for Efficiently Computing Three-Dimensional Folding Chains

Abstract Within the wide field of self-assembly, the self-folding chain has unique potential for reliable and repeatable assembly of three-dimensional structures as demonstrated by protein biosynthesis. This potential could be translated to self-reconfiguring robots by utilizing magnetic forces between the chain components as a driving force for the folding process. Due to the constraints introduced by the joints between the chain components, simulation of the dynamics of longer chains is computationally intensive and challenging. This article presents a novel analytical approach to formulate the Newton–Euler dynamics of a self-reconfiguring chain in a single vectorized differential equation. The vectorized differential equation allows for a convenient implementation of a parallel processing architecture using single instruction multiple data (SIMD) or graphical processing unit (GPU) computation and as a result can improve simulation time of rigid body chains. Properties of existing interpretations of the Newton–Euler and Euler–Lagrange algorithms are discussed in their efficiency to compute the dynamics of rigid body chains. Finally, GPU and SIMD-supported simulation, based on the vectorized Newton–Euler equations described in this article, are compared, showing a significant improvement in computation time using GPU architecture for long chains with certain chain geometry.

Vector Operations for Accelerating Expensive Bayesian Computations – A Tutorial Guide

Many applications in Bayesian statistics are extremely computationally intensive. However, they are often inherently parallel, making them prime targets for modern massively parallel processors. Multi-core and distributed computing is widely applied in the Bayesian community, however, very little attention has been given to fine-grain parallelisation using single instruction multiple data (SIMD) operations that are available on most modern CPUs. In this work, we practically demonstrate, using standard programming libraries, the utility of the SIMD approach for several topical Bayesian applications. Using the C programming language, we show that SIMD can improve the single-core floating point arithmetic performance by up to a factor of 6× compared scalar C code and more than 25× compared with optimised R code. Such improvements are multiplicative to any gains achieved through multi-core processing. We illustrate the potential of SIMD for accelerating Bayesian computations and provide the reader with techniques for exploiting modern massively parallel processing environments.

PEPFL: A framework for a practical and efficient privacy-preserving federated learning

As an emerging joint learning model, federated learning is a promising way to combine model parameters of different users for training and inference without collecting users’ original data. However, a practical and efficient solution has not been established in previous work due to the absence of efficient matrix computation and cryptography schemes in the privacy-preserving federated learning model, especially in partially homomorphic cryptosystems. In this paper, we propose a Practical and Efficient Privacy-preserving Federated Learning (PEPFL) framework. First, we present a lifted distributed ElGamal cryptosystem for federated learning, which can solve the multi-key problem in federated learning. Secondly, we develop a Practical Partially Single Instruction Multiple Data (PSIMD) parallelism scheme that can encode a plaintext matrix into single plaintext for encryption, improving the encryption efficiency and reducing the communication cost in partially homomorphic cryptosystem. In addition, based on the Convolutional Neural Network (CNN) and the designed cryptosystem, a novel privacy-preserving federated learning framework is designed by using Momentum Gradient Descent (MGD). Finally, we evaluate the security and performance of PEPFL. The experiment results demonstrate that the scheme is practicable, effective, and secure with low communication and computation costs.

An ASIP for Neural Network Inference on Embedded Devices with 99% PE Utilization and 100% Memory Hidden under Low Silicon Cost.

The computation efficiency and flexibility of the accelerator hinder deep neural network (DNN) implementation in embedded applications. Although there are many publications on deep neural network (DNN) processors, there is still much room for deep optimization to further improve results. Multiple dimensions must be simultaneously considered when designing a DNN processor to reach the performance limit of the architecture, including architecture decision, flexibility, energy efficiency, and silicon cost minimization. Flexibility is defined as the ability to support as many multiple networks as possible and to easily adjust the scale. For energy efficiency, there are huge opportunities for power efficiency optimization, which involves access minimization and memory latency minimization based on on-chip memory minimization. Therefore, this work focused on low-power and low-latency data access with minimized silicon cost. This research was implemented based on an ASIP (application specific instruction set processor) in which an ISA was based on the caffe2 inference operator and the hardware design was based on a single instruction multiple data (SIMD) architecture. The scalability and system performance of our SoC extension scheme were demonstrated. The VLIW was used to execute multiple instructions in parallel. All costs for data access time were thus eliminated for the convolution layer. Finally, the processor was synthesized based on TSMC 65 nm technology with a 200 MHz clock, and the Soc extension scheme was analyzed in an experimental model. Our design was tested on several typical neural networks, achieving 196 GOPS at 200 MHz and 241 GOPS/W on the VGG16Net and AlexNet.