Embedded Graphics Processing Unit Research Articles

Advanced hybrid MRAM based novel GPU cache system for graphic processing with high efficiency

With the rapid development of portable computing devices and users’ demand for high-quality graphics rendering, embedded Graphics Processing Units (GPU) systems for graphics processing are increasingly turning into a key component of computer architecture to enhance computability. The cache system based on traditional static random access memory (SRAM) plays a crucial role in GPUs. But high leakage, low lifetime and poor integration problems deeply plague the science and engineering field. In the paper, a novel magnetic random access memory (MRAM) based cache architecture of GPU systems is proposed for highly efficient graphics processing and computing accelerating, with the merits of high speed, long endurance, strong interference resistance, and ultra-low power consumption. Spin transfer torque-MRAM and spin orbit torque-MRAM are utilized in off-chip and on-chip caches, respectively. A controller design scheme with prefetching modules and optimized cache coherency protocols are adopted. After testing and evaluating with multiple loads, neural network models and datasets, the simulation results show that the proposed system can achieve up to 28%, 56%, and 66.45% optimizations mostly in terms of speed, energy and leakage power, respectively.

Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks

Safety-critical sensory applications, like medical diagnosis, demand accurate decisions from limited, noisy data. Bayesian neural networks excel at such tasks, offering predictive uncertainty assessment. However, because of their probabilistic nature, they are computationally intensive. An innovative solution utilizes memristors’ inherent probabilistic nature to implement Bayesian neural networks. However, when using memristors, statistical effects follow the laws of device physics, whereas in Bayesian neural networks, those effects can take arbitrary shapes. This work overcome this difficulty by adopting a variational inference training augmented by a “technological loss”, incorporating memristor physics. This technique enabled programming a Bayesian neural network on 75 crossbar arrays of 1,024 memristors, incorporating CMOS periphery for in-memory computing. The experimental neural network classified heartbeats with high accuracy, and estimated the certainty of its predictions. The results reveal orders-of-magnitude improvement in inference energy efficiency compared to a microcontroller or an embedded graphics processing unit performing the same task.

Sparsity Regularization-Based Real-Time Target Recognition for Side Scan Sonar with Embedded GPU

Side Scan Sonar (SSS) is widely used to search for seabed objects such as ships and wrecked aircraft due to its high-imaging-resolution and large planar scans. SSS requires an automatic real-time target recognition system to enhance search and rescue efficiency. In this paper, a novel target recognition method for SSS images in varied underwater environment, you look only once (YOLO)-slimming, based on convolutional a neural network (CNN) is proposed. The method introduces efficient feature encoders that strengthen the representation of feature maps. Channel-level sparsity regularization in model training is performed to speed up the inference performance. To overcome the scarcity of SSS images, a sonar image simulation method is proposed based on deep style transfer (ST). The performance on the SSS image dataset shows that it can reduce calculations and improves the inference speed with a mean average precision (mAP) of 95.3 and at least 45 frames per second (FPS) on an embedded Graphics Processing Unit (GPU). This proves its feasibility in practical application and has the potential to formulate an image-based real-time underwater target recognition system.

RadioML Meets FINN: Enabling Future RF Applications With FPGA Streaming Architectures

Deep neural networks (DNNs) are penetrating into a broad spectrum of applications and replacing manual algorithmic implementations, including the radio frequency communications domain with classical signal processing algorithms. However, the high throughput (gigasamples per second) and low latency requirements of this application domain pose a significant hurdle for adopting computationally demanding DNNs. In this article, we explore highly specialized DNN inference accelerator approaches on field-programmable gate arrays (FPGAs) for RadioML modulation classification. Using an automated end-to-end flow for the generation of the FPGA solution, we can easily explore a spectrum of solutions that optimize for different design targets, including accuracy, power efficiency, resources, throughput, and latency. By leveraging reduced precision arithmetic and customized streaming dataflow, we demonstrate a solution that meets the application requirements and outperforms alternative FPGA efforts by 3.5× in terms of throughput. Against modern embedded graphics processing units (GPUs), we measure $>\mathsf{10}\times$>10× higher throughput and $>\mathsf{100}\times$>100× lower latency under comparable accuracy and power envelopes.

TEGRAS: An Efficient Tegra Embedded GPU-Based RSA Acceleration Server

Industrial Internet of Things (IIoT) has strict requirements on the performance and security of devices. Public-key cryptography, as a kind of computing resource-consuming algorithm, is widely used in the digital signature, key exchange, and so on. The embedded graphics processing units (GPUs) are now rapidly achieving extraordinary computing power, such as NVIDIA Tegra K1/X1/X2/Xavier, which are also treated as edge computing devices. They are widely used in IIoT environments, such as intelligent manufacturing, smart cities, and vehicle-mounted systems. The performance advantages endow embedded GPUs with the possibility of accelerating cryptography that also requires high-density computing. This article implements an efficient Tegra-based embedded GPU RSA acceleration server-oriented IIoT, named TEGRAS. Various optimization methods are employed to promote efficiency, including multithreaded Montgomery multiplication and Chinese Remainder Theorem implementation on the resource-constricted embedded GPUs. With about 40–50 W of power consumption, TEGRAS can deliver 34 kops/s of RSA2048 signature generation and 1007 kops/s of RSA signature verification, which outperforms implementations in the desktop GPUs and embedded CPUs in the perspective of performance-to-power ratio. To evaluate TEGRAS in real-world scenarios, we additionally build a network stack to deliver digital signature services, which can provide more than 34 and 978 kops of signature generation and signature verification, respectively. In a word, based on the embedded GPU, we provide a high-throughput, low-latency, and ready-to-use RSA accelerator-oriented IIoT.

Distributed Real-Time Image Processing of Formation Flying SAR Based on Embedded GPUs

<b>Formation flying synthetic aperture radar (FF-SAR) systems, as an important development direction of multichannel SAR, can achieve high-resolution wide-swath (HRWS) imaging. Coherently combining data from satellite receivers puts a strain on traditional real-time processing systems based on individual satellites. Characteristics such as the power of real-time on-orbit processing platform must be properly balanced with constrained memory and parallel computational resources. This paper proposes a distributed SAR real-time imaging method based on embedded Graphics Processing Units (GPUs). The parallel computing method of chirp scaling (CS) algorithm is designed based on parallel programming model of compute unified device architecture (CUDA), and the optimization methods of memory and performance are proposed for the hardware architecture of embedded GPUs. In particular, unified memory management method is used to avoid data copying and communication delays between the CPU and GPU. A hardware verification system for distributed SAR real-time imaging processing based on multiple embedded GPUs is constructed. The proposed algorithm takes 5.86 seconds to process single-precision floating-point complex imaging with a data size of 8192 &#x00D7; 8192 on a single Jetson Nano platform. The actual power consumption is less than 5 W, and the performance-to-power ratio is greater than 1.7&#x0025;. The experimental results show that the real-time processing method based on embedded GPUs proposed in this paper has high performance and low power consumption.</b>

Real-time, High-resolution Depth Upsampling on Embedded Accelerators

High-resolution, low-latency apps in computer vision are ubiquitous in today’s world of mixed-reality devices. These innovations provide a platform that can leverage the improving technology of depth sensors and embedded accelerators to enable higher-resolution, lower-latency processing for 3D scenes using depth-upsampling algorithms. This research demonstrates that filter-based upsampling algorithms are feasible for mixed-reality apps using low-power hardware accelerators. The authors parallelized and evaluated a depth-upsampling algorithm on two different devices: a reconfigurable-logic FPGA embedded within a low-power SoC; and a fixed-logic embedded graphics processing unit. We demonstrate that both accelerators can meet the real-time requirements of 11 ms latency for mixed-reality apps. 1

드론의 변전소 애자 이상현상 탐지를 위한 심층신경망 경량화 모델 기반 알고리즘 개발 연구

This paper proposes a method to detect defected insulators of electrical substations in real-time on an embedded graphics processing unit (GPU) by using a drone camera. The proposed algorithm is based on a compressed deep neural network to detect defected insulators with a corona effect arising from an overcurrent. A compressed deep neural network model is developed to reduce the computation time while maintaining high detection accuracy for the embedded GPU board. This paper applies the MobileNetv2 structure and compression technique, a residual block, on YOLOv3 to reduce the overall memory size and calculation time of the detection architecture. The proposed network is trained with an image data set of a mockup insulator with and without the corona effect. The performance evaluation shows that the algorithm is able to detect the object with an accuracy of 85% at an average speed of 40 FPS on a NVIDIA Xavier board. In addition, the results show that the proposed detection network uses half the memory and less computation time than YOLOv3-Tiny.

Fast Reconstruction of 3D Point Cloud Model Using Visual SLAM on Embedded UAV Development Platform

In recent years, the rapid development of unmanned aerial vehicle (UAV) technologies has made data acquisition increasingly convenient, and three-dimensional (3D) reconstruction has emerged as a popular subject of research in this context. These 3D models have many advantages, such as the ability to represent realistic scenes and a large amount of information. However, traditional 3D reconstruction methods are expensive, and require long and complex processing. As a result, they cannot rapidly respond when used in time-sensitive applications, e.g., those for such natural disasters as earthquakes, debris flow, etc. Computer vision-based simultaneous localization and mapping (SLAM) along with hardware development based on embedded systems, can provide a solution to this problem. Based on an analysis of the principle and implementation of the visual SLAM algorithm, this study proposes a fast method to quickly reconstruct a dense 3D point cloud model on a UAV platform combined with an embedded graphics processing unit (GPU). The main contributions are as follows: (1) to resolve the contradiction between the resource limitations and the computational complexity of visual SLAM on UAV platforms, the technologies needed to compute resource allocation, communication between nodes, and data transmission and visualization in an embedded environment were investigated to achieve real-time data acquisition and processing. Visual monitoring to this end is also designed and implemented. (2) To solve the problem of time-consuming algorithmic processing, a corresponding parallel algorithm was designed and implemented based on the parallel programming framework of the compute unified device architecture (CUDA). (3) The visual odometer and methods of 3D “map” reconstruction were designed using under a monocular vision sensor to implement the prototype of the fast 3D reconstruction system. Based on preliminary results of the 3D modeling, the following was noted: (1) the proposed method was feasible. By combining UAV, SLAM, and parallel computing, a simple and efficient 3D reconstruction model of an unknown area was obtained for specific applications. (2) The parallel SLAM algorithm used in this method improved the efficiency of the SLAM algorithm. On the one hand, the SLAM algorithm required 1/6 of the time taken by the structure-from-motion algorithm. On the other hand, the speedup obtained using the parallel SLAM algorithm based on the embedded GPU on our test platform was 7.55 × that of the serial algorithm. (3) The depth map results show that the effective pixel with an error less than 15cm is close to 60%.

Work-Efficient Parallel Non-Maximum Suppression Kernels

Abstract In the context of object detection, sliding-window classifiers and single-shot convolutional neural network (CNN) meta-architectures typically yield multiple overlapping candidate windows with similar high scores around the true location of a particular object. Non-maximum suppression (NMS) is the process of selecting a single representative candidate within this cluster of detections, so as to obtain a unique detection per object appearing on a given picture. In this paper, we present a highly scalable NMS algorithm for embedded graphics processing unit (GPU) architectures that is designed from scratch to handle workloads featuring thousands of simultaneous detections on a given picture. Our kernels are directly applicable to other sequential NMS algorithms such as FeatureNMS, Soft-NMS or AdaptiveNMS that share the inner workings of the classic greedy NMS method. The obtained performance results show that our parallel NMS algorithm is capable of clustering 1024 simultaneous detected objects per frame in roughly 1 ms on both Tegra X1 and Tegra X2 on-die GPUs, while taking 2 ms on Tegra K1. Furthermore, our proposed parallel greedy NMS algorithm yields a 14–40x speed up when compared to state-of-the-art NMS methods that require learning a CNN from annotated data.

Accelerating computed tomographic imaging spectrometer reconstruction using a parallel algorithm exploiting spatial shift-invariance

Computed tomographic imaging spectrometers capture hyperspectral images in realtime. However, postprocessing the imagery can require enormous computational resources; thus, limiting its application to nonrealtime scenarios. To overcome these challenges, we developed a highly parallelizable algorithm that exploits spatial shift-invariance. To demonstrate the versatility of our algorithm, we developed implementations on a desktop and an embedded graphics processing unit. To our knowledge, our results show the fastest image reconstruction times reported.

Deep Learning Framework for Vehicle and Pedestrian Detection in Rural Roads on an Embedded GPU

Object detection, one of the most fundamental and challenging problems in computer vision. Nowadays some dedicated embedded systems have emerged as a powerful strategy for deliver high processing capabilities including the NVIDIA Jetson family. The aim of the present work is the recognition of objects in complex rural areas through an embedded system, as well as the verification of accuracy and processing time. For this purpose, a low power embedded Graphics Processing Unit (Jetson Nano) has been selected, which allows multiple neural networks to be run in simultaneous and a computer vision algorithm to be applied for image recognition. As well, the performance of these deep learning neural networks such as ssd-mobilenet v1 and v2, pednet, multiped and ssd-inception v2 has been tested. Moreover, it was found that the accuracy and processing time were in some cases improved when all the models suggested in the research were applied. The pednet network model provides a high performance in pedestrian recognition, however, the sdd-mobilenet v2 and ssd-inception v2 models are better at detecting other objects such as vehicles in complex scenarios.

A flexible and configurable GPGPU stereo matching framework

This paper presents a practical framework that consists of various local to global stereo matching algorithms on a general purpose computing on graphics processing units platform. The flexible framework provides users with a selection of individual sub-algorithms in each step of stereo matching. The framework runs on either a central processing unit or graphic processing unit across three different platforms, including a smartphone, embedded board, and desktop. On the basis of the proposed framework, we investigate different combinations of stereo matching algorithms for specific use cases. Accordingly, we provide the framework’s quantitative speed and accuracy analysis evaluated on the widely used stereo dataset. In addition, this paper also addresses the parallelization strategy on an embedded graphics processing unit. The experimental results show that the proposed framework is capable of real-time and accurate depth estimation of stereo input in video graphics array resolution on an embedded graphics processing unit.

A prototype system of portable laser speckle imager based on embedded graphics processing unit platform.

Laser speckle contrast imaging (LSCI) is a high-resolution full-field optical technique for measuring blood flow, which has been widely used in clinical and biomedical research. However, most of the current LSCI instruments are bulky, limiting their application settings. In this work, we proposed a prototype system of portable laser speckle imager. Different from the desktop laser speckle systems that utilize personal computer (PC), our system was designed with embedded GPU system (Jetson TX2, NVIDIA, USA) and a LCD touch screen (16.5 × 12.4 cm in size, 380 g in weight). In-vivo experiments showed that the portable GPU-based system had comparable performance with our laboratory LSCI system. Such a portable LSCI imager could be potentially used in a situation that requires for easy operation and installation, such as intraoperative monitoring or bedside diagnosis.

Development and evaluation of a low-cost and smart technology for precision weed management utilizing artificial intelligence

Most conventional sprayers apply agrochemicals uniformly, despite the fact that distribution of weeds is typically patchy, resulting in wastage of valuable compounds, increased costs, crop damage risk, pest resistance to chemicals, environmental pollution and contamination of products. To reduce these negative impacts, a smart sprayer was designed and developed utilizing machine vision and artificial intelligence to distinguish target weeds from non-target objects (e.g. vegetable crops) and precisely spray on the desired target/location. Two different experimental scenarios were designed to simulate a vegetable field and to evaluate the smart sprayer’s performance. The first scenario contained artificial weeds (targets) and artificial plants (non-targets). The second and most challenging scenario contained real plants; portulaca weeds as targets, and sedge weeds and pepper plants as non-targets. Two different embedded graphics processing unit (GPU) were evaluated as the smart sprayer processing unit (for image processing and target detection). The more powerful GPU (NVIDIA GTX 1070 Ti) achieved an overall precision of 71% and recall of 78% (for plant detection and target spraying accuracy) on the most challenging scenario with real plants, and 91% accuracy and recall for the first scenario with artificial plants. The less powerful GPU (NVIDIA Jetson TX2) achieved an overall precision and recall of 90% and 89% respectively on the first scenario with artificial plants, and 59% and 44% respectively on the second scenario with real plants. Finally, an RTK GPS was connected to the smart sprayer and an algorithm was developed to automatically generate weed maps and visualize the collected data (after every application). This smart technology integrates a state of the art (AI-based) weed detection system, a novel fast and precision spraying system, and a weed mapping system. It can significantly reduce the quantity of agrochemicals required, especially compared with traditional broadcast sprayers that usually treat the entire field, resulting in unnecessary application to areas that do not require treatment. It could also reduce costs, risk of crop damage and excess herbicide residue, as well as potentially reduce environmental impact.

Massively parallel hybrid algorithm on embedded graphics processing unit for unmanned aerial vehicle path planning

To operate autonomously, military unmanned aerial vehicles (UAVs) must be equipped with a path planning module capable of calculating feasible trajectories. This is a highly complex and nonlinear optimisation problem that challenges state of the art methods. In this paper, we present a massively parallel hybrid algorithm to solve the path planning problem for fixed-wing military UAVs. The proposed solution combines the strengths of the genetic algorithm (GA) and the particle swarm optimisation and allows for the calculation of quasi-optimal paths in realistic 3D environments. To reduce the execution time, the proposed algorithm is parallelised on the NVIDIA Jetson TX1 embedded graphics processing unit (GPU). By exploiting the parallel architecture of the GPU, the runtime is reduced by a factor of 23.6× to just 4.3 seconds while requiring only 10 watts, making it an excellent solution for on-board path planning. The proposed system is tested in a simulation using 18 scenarios on six different terrains.

Massively parallel hybrid algorithm on embedded graphics processing unit for unmanned aerial vehicle path planning

To operate autonomously, military unmanned aerial vehicles (UAVs) must be equipped with a path planning module capable of calculating feasible trajectories. This is a highly complex and nonlinear optimisation problem that challenges state of the art methods. In this paper, we present a massively parallel hybrid algorithm to solve the path planning problem for fixed-wing military UAVs. The proposed solution combines the strengths of the genetic algorithm (GA) and the particle swarm optimisation and allows for the calculation of quasi-optimal paths in realistic 3D environments. To reduce the execution time, the proposed algorithm is parallelised on the NVIDIA Jetson TX1 embedded graphics processing unit (GPU). By exploiting the parallel architecture of the GPU, the runtime is reduced by a factor of 23.6× to just 4.3 seconds while requiring only 10 watts, making it an excellent solution for on-board path planning. The proposed system is tested in a simulation using 18 scenarios on six different terrains.

Embedded GPU implementation of sensor correction for on-board real-time stream computing of high-resolution optical satellite imagery

In recent years, the data generation rate of high-resolution optical satellites is increasing rapidly, and this situation brings massive stress to the data downloading link and data processing system. To relieve downloading link stress and enhance the time efficiency of information acquisition, on-board processing needs to be introduced. Current on-board solutions are mostly based on digital signal processing and field-programmable gate array, which are obturated, lack flexibility, and are highly costly to implement. This paper takes sensor correction, the prerequisite geometric step of high-resolution optical satellite data processing, as an example, using a simulation prototype made of a double-module data parallel pipeline system based on NVIDIA embedded graphics processing unit platform, and proposes a feasible stream computing approach for low power consumption, flexible, and expandable on-board real-time processing. The experiments use a short strip of GaoFen-9 satellite data that contain 4.5 standard scenes to validate the performance and correctness of this approach. Compared to the same algorithms on the Dell PowerEdge T630 Server, the on-board version demonstrates an obvious performance advantage. When considering the contrast of power consumption for each platform, the advantage becomes even more significant. With statistics and analysis of experimental results, the timeline of processing demonstrates that this approach could meet the on-board real-time sensor correction requirement. And correctness is also verified by the root-mean-square error of pixel-by-pixel image comparison experiments.

Model Predictive Control for Autonomous Navigation Using Embedded Graphics Processing Unit

Abstract The objective of this work is to implement a Model Predictive Control (MPC) algorithm on an embedded Graphics Processing Unit (GPU) card. A MPC model for the autonomous navigation of a ground mobile robot is proposed. GPU CUDA code implementation and CUDA optimization techniques are discussed for this specific problem. The GPU-accelerated application permits extending the prediction horizon and evaluating more future trajectories compared to usual time-constrained CPU implementations. Simulation results and a preliminary experiment are presented to demonstrate the efficiency of the real-time algorithm.

Priority-driven spatial resource sharing scheduling for embedded graphics processing units

Many visual tasks in modern personal devices such smartphones resort heavily to graphics processing units (GPUs) for their fluent user experiences. Because most GPUs for embedded systems are non-preemptive by nature, it is important to schedule GPU resources efficiently across multiple GPU tasks. We present a novel spatial resource sharing (SRS) technique for GPU tasks, called a budget-reservation spatial resource sharing (BR-SRS) scheduling, which limits the number of GPU processing cores for a job based on the priority of the job. Such a priority-driven resource assignment can prevent a high-priority foreground GPU task from being delayed by background GPU tasks. The BR-SRS scheduler is invoked only twice at the arrival and completion of jobs, and thus, the scheduling overhead is minimized as well. We evaluated the performance of our scheduling scheme in an Android-based smartphone, and found that the proposed technique significantly improved the performance of high-priority tasks in comparison to the previous temporal budget-based multi-task scheduling.