ARM CPU Research Articles

A pipelining strategy for accelerating convolution neural networks on ARM CPUs

AbstractConvolution is a primary operation in convolution neural networks. The speed of inference is mainly decided by the speed of the convolutional layer. Improving the performance of embedded processors makes it possible to process the inference on embedded devices. In this article, a pipelining strategy of single instruction and multiple data (SIMD) instructions is proposed to finely optimize the process of the 3 × 3 convolution on ARM‐based CPUs. We implement the SIMD group to improve the efficiency of the SIMD pipeline. A tiling method is exploited to increase data reuse during the process. An evaluation model is proposed to guide the design of the tiling method and register allocation. The speed of our implementation is 5.18 times of the GNU compiler collection compiled unoptimized version on RK3288. The effect of our optimizing method is measured by a performance profiling tool, the performance information suggests that the pipelining strategy has a significant effect for both normal and depthwise separable convolution. By implementing multithread processing, the speedup achieves 18.3 compared with the single thread unoptimized version.

ReS2tAC-UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices.

With the emergence of low-cost robotic systems, such as unmanned aerial vehicle, the importance of embedded high-performance image processing has increased. For a long time, FPGAs were the only processing hardware that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded systems. However, the recently increasing availability of embedded GPU-based systems, such as the NVIDIA Jetson series, comprised of an ARM CPU and a NVIDIA Tegra GPU, allows for massively parallel embedded computing on graphics hardware. With this in mind, we propose an approach for real-time embedded stereo processing on ARM and CUDA-enabled devices, which is based on the popular and widely used Semi-Global Matching algorithm. In this, we propose an optimization of the algorithm for embedded CUDA GPUs, by using massively parallel computing, as well as using the NEON intrinsics to optimize the algorithm for vectorized SIMD processing on embedded ARM CPUs. We have evaluated our approach with different configurations on two public stereo benchmark datasets to demonstrate that they can reach an error rate as low as 3.3%. Furthermore, our experiments show that the fastest configuration of our approach reaches up to 46 FPS on VGA image resolution. Finally, in a use-case specific qualitative evaluation, we have evaluated the power consumption of our approach and deployed it on the DJI Manifold 2-G attached to a DJI Matrix 210v2 RTK unmanned aerial vehicle (UAV), demonstrating its suitability for real-time stereo processing onboard a UAV.

Embedded real-time infrared and visible image fusion for UAV surveillance

Infrared and visible image fusion is a beneficial processing task for Unmanned Aerial Vehicle (UAV) surveillance, which can improve visibility by combining the advantages of the infrared camera and the visible light camera. An embedded onboard solution is necessary for UAV-based surveillance missions because it reduces the amount of data that are transmitted to the ground. In this paper, we propose an infrared and visible light image fusion method and implement it on two platforms with commonly used HW accelerators for embedded vision applications: Zedboard (ARM + FPGA) and NVIDIA TX1 (ARM + GPU), and compare their performances. To verify the usefulness of image fusion, we carry out sufficient experiments to prove that image fusion can improve the target detection ability of a UAV in different scenes. The detection rate for target detection is up to 0.926 in our experiments. The execution times on the ZedBoard and the TX1 are, respectively, 205.3 FPS and 36.6 FPS (38 $$\times$$ and 6.7 $$\times$$ in comparison to an ARM Cortex-A9 processor). Our results also show that the ZedBoard achieves an energy/frame reduction ratio of 7.1 $$\times$$ and 18.9 $$\times$$ respectively compared to the TX1 and the ARM CPU. This work is based on a UAV platform designed by ourselves, and all image sets are real scenes that we have captured. This demonstrates that the proposed method is viable and reflects the actual needs of real UAV surveillance systems.

SplitSR

Super-resolution (SR) is a coveted image processing technique for mobile apps ranging from the basic camera apps to mobile health. Existing SR algorithms rely on deep learning models with significant memory requirements, so they have yet to be deployed on mobile devices and instead operate in the cloud to achieve feasible inference time. This shortcoming prevents existing SR methods from being used in applications that require near real-time latency. In this work, we demonstrate state-of-the-art latency and accuracy for on-device super-resolution using a novel hybrid architecture called SplitSR and a novel lightweight residual block called SplitSRBlock. The SplitSRBlock supports channel-splitting, allowing the residual blocks to retain spatial information while reducing the computation in the channel dimension. SplitSR has a hybrid design consisting of standard convolutional blocks and lightweight residual blocks, allowing people to tune SplitSR for their computational budget. We evaluate our system on a low-end ARM CPU, demonstrating both higher accuracy and up to 5× faster inference than previous approaches. We then deploy our model onto a smartphone in an app called ZoomSR to demonstrate the first-ever instance of on-device, deep learning-based SR. We conducted a user study with 15 participants to have them assess the perceived quality of images that were post-processed by SplitSR. Relative to bilinear interpolation --- the existing standard for on-device SR --- participants showed a statistically significant preference when looking at both images (Z=-9.270, p<0.01) and text (Z=-6.486, p<0.01).

Collaborative execution of fluid flow simulation using non-uniform decomposition on heterogeneous architectures

The demand for computing power, along with the diversity of computational problems, culminated in a variety of heterogeneous architectures. Among them, hybrid architectures combine different specialized hardware into a single chip, comprising a System-on-Chip (SoC). Since these architectures usually have limited resources, efficiently splitting data and tasks between the different hardware is primal to improve performance. In this context, we explore the non-uniform decomposition of the data domain to improve fluid flow simulation performance on heterogeneous architectures. We evaluate two hybrid architectures: one comprised of a general-purpose x86 CPU and a graphics processing unit (GPU) integrated into a single chip (AMD Kaveri SoC), and another comprised by a general-purpose ARM CPU and a Field Programmable Gate Array (FPGA) integrated into the same chip (Intel Arria 10 SoC). We investigate the effects on performance and energy efficiency of data decomposition on each platform’s devices on a collaborative execution. Our case study is the well-known Lattice Boltzmann Method (LBM), where we apply the technique and analyze the performance and energy efficiency of five kernels on both devices on each platform. Our experimental results show that non-uniform partitioning improves the performance of LBM kernels by up to 11.40% and 15.15% on AMD Kaveri and Intel Arria 10, respectively. While AMD’s Kaveri platform’s performance efficiency is of up to 10.809 MLUPS with an energy efficiency of 142.881 MLUPKJ, Intel’s Arria 10 platform’s is of up to 1.12 MLUPS and 82.272 MLUPKJ.

An implementation methodology for Neural Network on a Low-end FPGA Board

Artificial Intelligence(AI) has achieved unprecedented success in various fields that include image, speech, or even video recognition. Most systems are implemented on power-hungry devices like CPU, GPU, or even TPU to process data due to the models' high computation and storage complexity. CPU platforms do weak in computation capacity, while energy budgets and expense of GPU and TPU are often not affordable to edge computing in the industrial business. Recently, the FPGA-based Neural Network (NN) accelerator has been a trendy topic in the research field. It is regarded as a promising solution to suppress GPU in both speed and energy efficiency with its specifically designed architecture. Our work performs on a low-end FPGA board, a more desirable platform in meeting the restrictions of energy efficiency and computational resource on an autonomous driving car. We propose a methodology that integrates a NN model into the board using HLS description in this paper. The whole design consists of algorithm-level downscaling and hardware optimization. The former emphasizes the model downscale through model pruning and binarization, which balance the model size and accuracy. The latter applies various HLS design techniques on each NN component, like loop unrolling, inter- /intra- level pipelining, and so on, to speed-up the application running on the target board. In the case study of tiny YOLO (You Only Look Once) v3, the model running on PYNQ-Z1 presents up to 22x acceleration comparing with the PYNQ's ARM CPU. Energy efficiency also achieves 3x better than Xeon E5-2667. To verify the flexibility of our methodology, we extend our work to the BinaryConnect and DoReFaNet. It is worth mentioning that the BinaryConnect even achieves around 100x acceleration comparing with it purely running on the PYNQ-Z1 ARM core.

A data-driven approach to run agent-based multi-modal traffic simulations on heterogeneous CPU-GPU hardware

In order to keep short and acceptable run times of agent-based mobility simulators that are used for scenarios, which are of increasing complexity and scale, there is need for increased computational efficiency. While, this need may be addressed by the use of heterogeneous hardware, existing traffic models may be inefficient or not run on such hardware. To simplify the development of mobility simulators with support for heterogeneous hardware, we propose a novel data-driven approach in which the data layer is built such that multiple types of hardware can yield improved run time performance. Using this novel approach, we port an existing GPU-accelerated, large-scale, multi-modal, mobility simulator to modern many-core CPUs. While, a CPU backend runs 3.89 times slower compared to a GPU backend, the CPU backend is 11.64 times faster than another widely used agent-based simulator. Moreover, the run time of the CPU backend on ARM CPUs is comparable to the run time on x86 CPUs.

New parallel computing algorithm of molecular dynamics for extremely huge scale biological systems.

In this paper, we address high performance extreme-scale molecular dynamics (MD) algorithm in the GENESIS software to perform cellular-scale molecular dynamics (MD) simulations with more than 100,000 CPU cores. It includes (1) the new algorithm of real-space nonbonded interactions maximizing the performance on ARM CPU architecture, (2) reciprocal-space nonbonded interactions minimizing communicational cost, (3) accurate temperature/pressure evaluations that allows a large time step, and (4) effective parallel file inputs/outputs (I/O) for MD simulations of extremely huge systems. The largest system that contains 1.6 billion atoms was simulated using MD with a performance of 8.30 ns/day on Fugaku supercomputer. It extends the available size and time of MD simulations to answer unresolved questions of biomacromolecules in a living cell.

SEPAR: A New Lightweight Hybrid Encryption Algorithm with a Novel Design Approach for IoT

This paper presents a new hybrid encryption algorithm with 16-bit block size and a 128-bit initialization vector, referred to as SEPAR, and it is suitable for IoT devices. The design idea of this algorithm combines pseudorandom permutation and pseudorandom generator functions. This smart integration causes resistance improvement against common cryptographic attacks meanwhile leads to cipher speed increment. Investigation of security analysis on the algorithm and results of the NIST statistical test suit proves its resistance against common cryptographic attacks as linear and differential cryptanalysis. Furthermore, efficient software implementation of SEPAR is presented on 8, 16 and 32-bit platforms. Compared to BORON cipher, SEPAR provides 42.22% throughput improvement on 32-bit ARM CPU. Also, for 8-bit and 16-bit microcontroller, SEPAR provides 87.91% and 98.01% performance improvements compared to present, respectively.

ThunderX2 Performance and Energy-Efficiency for HPC Workloads

In the last years, the energy efficiency of HPC systems is increasingly becoming of paramount importance for environmental, technical, and economical reasons. Several projects have investigated the use of different processors and accelerators in the quest of building systems able to achieve high energy efficiency levels for data centers and HPC installations. In this context, Arm CPU architecture has received a lot of attention given its wide use in low-power and energy-limited applications, but server grade processors have appeared on the market just recently. In this study, we targeted the Marvell ThunderX2, one of the latest Arm-based processors developed to fit the requirements of high performance computing applications. Our interest is mainly focused on the assessment in the context of large HPC installations, and thus we evaluated both computing performance and energy efficiency, using the ERT benchmark and two HPC production ready applications. We finally compared the results with other processors commonly used in large parallel systems and highlight the characteristics of applications which could benefit from the ThunderX2 architecture, in terms of both computing performance and energy efficiency. Pursuing this aim, we also describe how ERT has been modified and optimized for ThunderX2, and how to monitor power drain while running applications on this processor.

Energy-Efficient Acceleration of Deep Neural Networks on Realtime-Constrained Embedded Edge Devices

This paper presents a hardware management technique that enables energy-efficient acceleration of deep neural networks (DNNs) on realtime-constrained embedded edge devices. It becomes increasingly common for edge devices to incorporate dedicated hardware accelerators for neural processing. The execution of neural accelerators in general follows a host-device model, where CPUs offload neural computations (e.g., matrix and vector calculations) to the accelerators for datapath-optimized executions. Such a serialized execution is simple to implement and manage, but it is wasteful for the resource-limited edge devices to exercise only a single type of processing unit in a discrete execution phase. This paper presents a hardware management technique named NeuroPipe that utilizes heterogeneous processing units in an embedded edge device to accelerate DNNs in energy-efficient manner. In particular, NeuroPipe splits a neural network into groups of consecutive layers and pipelines their executions using different types of processing units. The proposed technique offers several advantages to accelerate DNN inference in the embedded edge device. It enables the embedded processor to operate at lower voltage and frequency to enhance energy efficiency while delivering the same performance as uncontrolled baseline executions, or inversely it can dispatch faster inferences at the same energy consumption. Our measurement-driven experiments based on NVIDIA Jetson AGX Xavier with 64 tensor cores and eight-core ARM CPU demonstrate that NeuroPipe reduces energy consumption by 11.4% on average without performance degradation, or it can achieve 30.5% greater performance for the same energy consumption.

Heterogeneous Real-Time Co-Emulation for Communication-Enabled Global Control of AC/DC Grid Integrated With Renewable Energy

The information, and communication technologies (ICTs) are increasingly merging with the conventional power systems. For the design, and development of modern AC/DC grids with integrated renewable energy sources, the system-level control schemes with ICTs involved should be evaluated in a co-simulation framework. In this work, a heterogeneous hardware real-time co-emulator composed of FPGAs, many-core GPU, and multi-core CPU devices is proposed to study the communication-enabled global control schemes of hybrid AC/DC networks. The electromagnetic transient (EMT) power system emulation is conducted on the Xilinx FPGA boards to provide nearly continuous instantaneous waveforms for cyber layer sampling; the communication layer is simulated on the ARM CPU cores of the embedded NVIDIA Jetson platform for flexible computing, and programming;, and the control functions for modular multi-level converters are executed on GPU cores of the Jetson platform for parallel calculation. The data exchange between FPGAs, and Jetson is achieved via the PCI express interface, which simulates the sampling operation of the AC phasor measurement unit (PMU), and DC merging unit (DC-MU). The power overflow, and DC fault cases are investigated to demonstrate the validity, and effectiveness of the proposed co-emulation hardware architecture, and global control schemes.

First Steps in Porting the LFRic Weather and Climate Model to the FPGAs of the EuroExa Architecture

In recent years, there has been renewed interest in the use of field-programmable gate arrays (FPGAs) for high-performance computing (HPC). In this paper, we explore the techniques required by traditional HPC programmers in porting HPC applications to FPGAs, using as an example the LFRic weather and climate model. We report on the first steps in porting LFRic to the FPGAs of the EuroExa architecture. We have used Vivado High-Level Syntheusywwi to implement a matrix-vector kernel from the LFRic code on a Xilinx UltraScale+ development board containing an XCZU9EG multiprocessor system-on-chip. We describe the porting of the code, discuss the optimization decisions, and report performance of 5.34 Gflop/s with double precision and 5.58 Gflop/s with single precision. We discuss sources of inefficiencies, comparisons with peak performance, comparisons with CPU and GPU performance (taking into account power and price), comparisons with published techniques, and comparisons with published performance, and we conclude with some comments on the prospects for future progress with FPGA acceleration of the weather forecast model. The realization of practical exascale-class high-performance computinems requires significant improvements in the energy efficiency of such systems and their components. This has generated interest in computer architectures which utilize accelerators alongside traditional CPUs. FPGAs offer huge potential as an accelerator which can deliver performance for scientific applications at high levels of energy efficiency. The EuroExa project is developing and building a high-performance architecture based upon ARM CPUs with FPGA acceleration targeting exascale-class performance within a realistic power budget.

Exhaustive single bit fault analysis. A use case against Mbedtls and OpenSSL’s protection on ARM and Intel CPU

Faults in software implementations target both data and instructions at different locations. Bellcore attack is a well-known fault attack that is able to break CRT-RSA. In response, cryptographic libraries such as OpenSSL are designed with protection. In this paper, new faults locations are shown on OpenSSL implementation of the CRT-RSA signature running on Intel and ARM processors. Among those faults, one restores the Bellcore attack on the OpenSSL library despite a protection and another is a complete new fault that exploits the OpenSSL protection to get RSA private key. Quite surprisingly, one of the exhibited faults is, ironically enough, made possible because of the existence of such protection. Mbedtls library is also analyzed in this paper. A way to find all exploitable faults on monobit flip fault model is also detailed.

ARM-VO: an efficient monocular visual odometry for ground vehicles on ARM CPUs

Localization is among the most important prerequisites for autonomous navigation. Vision-based systems have got great attention in recent years due to numerous camera advantages over other sensors. Reducing the computational burden of such systems is an active research area making them applicable to resource-constrained systems. This paper aims to propose and compare a fast monocular approach, named ARM-VO, with two state-of-the-art algorithms, LibViso2 and ORB-SLAM2, on Raspberry Pi 3. The approach is a sequential frame-to-frame scheme that extracts a sparse set of well-distributed features and tracks them in upcoming frames using Kanade–Lucas–Tomasi tracker. A robust model selection is used to avoid degenerate cases of fundamental matrix. Scale ambiguity is resolved by incorporating known camera height above ground. The method is open-sourced [ https://github.com/zanazakaryaie/ARM-VO ] and implemented in ROS mostly using NEON C intrinsics while exploiting the multi-core architecture of the CPU. Experiments on KITTI dataset showed that ARM-VO is 4–5 times faster and is the only method that can work almost real-time on Raspberry Pi 3. It achieves significantly better results than LibViso2 and is ranked second after ORB-SLAM2 in terms of accuracy.

Special-purpose computer for electroholography in embedded systems

Electroholography is attracting attention as an ideal 3D image presentation method. Developing such methods involves significant computational complexity; thus, they are difficult to realize. Typically, large cluster systems with a control PC and a field-programmable gate array (FPGA) acceleration board are used to reduce hologram calculation time. However such systems cannot be used for embedded 3D systems, such as a head-mounted display. In this study, we developed a compact holographic computer using a Xilinx Zynq UltraScale+ MPSoC with an ARM CPU and an FPGA on a single chip. The proposed system can reproduce a 3D video at 15 frames per second for a spatial light modulator of 1,920 $\times$× 1,080 pixels from an object represented by 6,500 point cloud sources. The proposed system can be embedded in various systems, e.g., head-mounted displays and 3D televisions.

Hardware Accelerator for Proof-Of-Work Operation in IOTA Cryptocurrency

The authors proposed hardware accelerator for the proof-of-work (PoW) operation in IOTA cryptocurrency. IOTA allows making secure and authenticated quantum resistant channels between IoT devices for communication and micropayments with no fees. Hardware acceleration reduces transaction time and increases throughput. In the article authors consider a basic theory of IOTA operations, generating and signing transaction with Winternitz one-time signatures. The authors describe operation principle of a new ternary hash function Curl. Winternitz one-time signatures and ternary hash function make IOTA quantum resistant. The core of IOTA is called Tangle and unlike Blockchain has Directed Acyclic Graph structure. There are no miners in the Tangle and IoT devices themselves maintain network operation, which leads to unlimited scalability and absence of fees. To add new transaction to the Tangle, IoT devices need to perform PoW operation for spam and Sybil attack protection — iteratively calculate Curl hash function for the IOTA transaction and change nonce field of the transaction until obtained result doesn’t satisfy given criteria, which is some amount of consecutive zero ternary values at the end of transaction hash. The software implementation of Curl hash function is very slow, the PoW operation on embedded devices can last up to 50 minutes, so the hardware acceleration of PoW operation is relevant task. In the proposed work authors created hardware accelerator for IOTA PoW operation. The structure and operation principle of accelerator is described. The proof-of-concept implementations was launched on DE10-nano board, based on Intel programmable logic chip. The proposed PoW hardware accelerator has parameterizable structure. It is possible manually set the number of PoW computing units by changing parameter value. In such parameterizable system one PoW computing unit is master and all remaining PoW units are slaves. Master PoW unit absorbs IOTA transaction, except nonce part, to midstate register utilizing sponge-like approach. Then all POW computing units (master and slaves) preload own state registers from midstate, randomly change personal nonces and start iterative search of valid nonce. When one of PoW computing units finds a valid nonce, PoW operation ends, nonce stored to destination buffer in SDRAM and interrupt is generated for ARM CPU. Final implementation of IOTA PoW hardware accelerator for DE10-nano board contains 11 PoW computing units, delivers 13.2 MH/s hash rate and gives x1000 speedup, compared to software implementation from IOTA developers, for only 30% of 5CSEBA6U23I7 programmable logic chip resources at 100 MHz clock frequency. The average PoW computation time in such implementation is 0.8 second.Ref. 14, img. 6.

A performance analysis of the first generation of HPC‐optimized Arm processors

SummaryIn this paper, we present performance results from Isambard, the first production supercomputer to be based on Arm CPUs that have been optimized specifically for HPC. Isambard is the first Cray XC50 “Scout” system, combining Cavium ThunderX2 Arm‐based CPUs with Cray's Aries interconnect. The full Isambard system will be delivered in the summer of 2018, when it will contain over 10 000 Arm cores. In this work, we present node‐level performance results from eight early‐access nodes that were upgraded to B0 beta silicon in March 2018. We present node‐level benchmark results comparing ThunderX2 with mainstream CPUs, including Intel Skylake and Broadwell, as well as Xeon Phi. We focus on a range of applications and mini‐apps important to the UK national HPC service, ARCHER, as well as to the Isambard project partners and the wider HPC community. We also compare performance across three major software toolchains available for Arm: Cray's CCE, Arm's version of Clang/Flang/LLVM, and GNU.

Low-Cost Low-Power Acceleration of a Microwave Imaging Algorithm for Brain Stroke Monitoring

Microwave imaging can effectively image the evolution of a hemorrhagic stroke thanks to the dielectric contrast between the blood and the surrounding brain tissues. To keep low both the form factor and the power consumption in a bedside device, we propose implementing a microwave imaging algorithm for stroke monitoring in a programmable system-on-chip, in which a simple ARM-based CPU offloads to an FPGA the heavy part of the computation. Compared to a full-software implementation in the ARM CPU, we obtain a 5× speed increase with hardware acceleration without loss in accuracy and precision.

Data Exfiltration from Air-Gapped Computers based on ARM CPU

Air-gapped Network is a network isolated from public networks. Several techniques of data exfiltration from air-gapped networks have been recently proposed. Air-gap malware is a malware that breaks the isolation of an air-gapped computer using air-gap covert channels, which extract information from air-gapped computers running on air-gap networks. Guri et al. presented an air-gap malware “GSMem”, which can exfiltrate data from air-gapped computers over GSM frequencies, 850 MHz to 900MHz. GSMem makes it possible to send data using the radio waves leaked out from the system bus between CPU and RAM. It generates binary amplitude shift keying (B-ASK) modulated waves with x86 SIMD instruction. In order to efficiently emit electromagnetic waves from the system-bus, it is necessary to access the RAM without being affected by the CPU caches. GSMem adopts an instruction that writes data without accessing CPU cache in Intel CPU. This paper proposes an air-gap covert channel for computers based on ARM CPU, which includes a software algorithm that can effectively cause cache misses. It is also a technique to use NEON instructions and transmit B-ASK modulated data by radio waves radiated from ARM based computer (e.g. Raspberry Pi 3). The experiment shows that the proposed program sends binary data using radio waves (about 1000kHz ~ 1700kHz) leaked out from system-bus between ARM CPU and RAM. The program can also run on Android machines based on ARM CPU (e.g. ASUS Zenpad 3S 10 and OnePlus 3).