Mobile GPU Research Articles

Progressive ray casting for volumetric models on mobile devices

Mobile devices have experienced an incredible market penetration in the last decade. Currently, medium to premium smartphones are relatively affordable devices. With the increase in screen size and resolution, together with the improvements in performance of mobile CPUs and GPUs, more tasks have become possible. In this paper we explore the rendering of medium to large volumetric models on mobile and low performance devices in general. To do so, we present a progressive ray casting method that is able to obtain interactive frame rates and high quality results for models that not long ago were only supported by desktop computers.

Toward High Mobile GPU Performance Through Collaborative Workload Offloading

The ever increasing of display resolution on mobile devices raises high demand for GPU rendering details. However, the challenge of poor hardware support but fine-grained rendering details often makes user unsatisfied especially in calling for high frame rate scenarios, e.g., game. To resolve such issue, we propose ButterFly, a novel system which collaboratively utilizes mobile GPUs to process high-quality rendering details for on-the-go mobile users. In particular, ButterFly achieves three technical contributions for the collaborative design: (1) a mobile device can migrate GPU workloads in buffer queue to peers, (2) the collaborative rendering mechanism benefits user high quality details while significant power saving performance, and (3) unnecessary 3D texture rendering can be clipped for further optimization. All the techniques are compatible with the OpenGL ES standards. Furthermore, a 40-person survey perceives that ButterFly can provide excellent user experience of both rendering details and frame rate over Wi-Fi network. In addition, our comprehensive trace-driven experiments on Android prototype reveal the benefits of Butterfly have more superior performance over state-of-the-art systems, which achieves more than 28.3 percent power saving.

A Novel Performance Prediction Model for Mobile GPUs

With the fast-growing development of mobile devices, the application of high-end three-dimensional (3-D) graphics is expanding to include usage in mobile platforms. Recent mobile application processors are equipped with a multicore CPU and a mobile GPU on a single chip, thereby enabling the incorporation of high-end 3-D graphics into mobile devices. The problem is that such features consume large amounts of power. Thus, previous studies focused on estimating the power consumption of mobile GPUs, but such research has been unfamiliar with user perspective. To address this deficiency, the current work developed a novel performance prediction model for mobile GPUs on Adreno. The model uses both the instruction throughput of a unified shader and GFLOPS. The utilization of the Adreno GPUs was adjusted to its maximum value to ensure that their performance is unaffected by dynamic voltage and frequency scaling and throttling functions. The model was validated using GFXBench under real game application environments. The simulation results provided the computational rates of each hardware unit of the Adreno GPUs and the rate of increase in the instruction processing of the unified shader. To verify the accuracy of the model, we compared the difference rates of the prediction results between those derived from the proposed model and those using Snapdragon profiler. The average error rate was 3.32% with three applications running on four different mobile devices.

Z2 traversal order: An interleaving approach for VR stereo rendering on tile-based GPUs

With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential. Because VR stereo rendering has increased computational costs to separately render views for the left and right eyes, to reduce the rendering cost in VR applications, we present a novel traversal order for tile-based mobile GPU architectures: Z2 traversal order. In tile-based mobile GPU architectures, a tile traversal order that maximizes spatial locality can increase GPU cache efficiency. For VR applications, our approach improves upon the traditional Z order curve. We render corresponding screen tiles in left and right views in turn, or simultaneously, and as a result, we can exploit spatial adjacency of the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. Our experimental results show that Z2 traversal order can reduce external memory bandwidth requirements and increase rendering performance.

Efficient and robust deep networks for semantic segmentation

This paper explores and investigates deep convolutional neural network architectures to increase the efficiency and robustness of semantic segmentation tasks. The proposed solutions are based on up-convolutional networks. We introduce three different architectures in this work. The first architecture, called Part-Net, is designed to tackle the specific problem of human body part segmentation and to provide robustness to overfitting and body part occlusion. The second network, called Fast-Net, is a network specifically designed to provide the smallest computation load without losing representation power. Such an architecture is capable of being run on mobile GPUs. The last architecture, called M-Net, aims to maximize the robustness characteristics of deep semantic segmentation approaches through multiresolution fusion. The networks achieve state-of-the-art performance on the PASCAL Parts dataset and competitive results on the KITTI dataset for road and lane segmentation. Moreover, we introduce a new part segmentation dataset, the Freiburg City dataset, which is designed to bring semantic segmentation to highly realistic robotics scenarios. Additionally, we present results obtained with a ground robot and an unmanned aerial vehicle and a full system to explore the capabilities of human body part segmentation in the context of human–robot interaction.

Earliest Virtual Deadline Zero Laxity Scheduling for Improved Responsiveness of Mobile GPUs

Earliest virtual deadline zero laxity (EVDZL) algorithm is proposed for mobile GPU schedulers for its improved responsiveness. Responsiveness of user interface (UI) is one of the key factors in evaluating smart devices because of its significant impacts on user experiences. However, conventional GPU schedulers based on completely fair scheduling (CFS) shows a poor responsiveness due to its algorithmic complexity. In this letter, we present the EVDZL scheduler based on the conventional earliest deadline zero laxity (EDZL) algorithm by accommodating the virtual laxity concept into the scheduling. Experimental results show that the EVDZL scheduler improves the response time of the Android UI by 9.6% compared with the traditional CFS scheduler.

Parallel Digital Predistortion Design on Mobile GPU and Embedded Multicore CPU for Mobile Transmitters

Digital predistortion (DPD) is a widely adopted baseband processing technique in current radio transmitters. While DPD can effectively suppress unwanted spurious spectrum emissions stemming from imperfections of analog RF and baseband electronics, it also introduces extra processing complexity and poses challenges on efficient and flexible implementations, especially for mobile cellular transmitters, considering their limited computing power compared to basestations. In this paper, we present high data rate implementations of broadband DPD on modern embedded processors, such as mobile GPU and multicore CPU, by taking advantage of emerging parallel computing techniques for exploiting their computing resources. We further verify the suppression effect of DPD experimentally on real radio hardware platforms. Performance evaluation results of our DPD design demonstrate the high efficacy of modern general purpose mobile processors on accelerating DPD processing for a mobile transmitter.

CNN Design for Real-Time Traffic Sign Recognition

Nowadays, more and more object recognition tasks are being solved with Convolutional Neural Networks (CNN). Due to its high recognition rate and fast execution, the convolutional neural networks have enhanced most of computer vision tasks, both existing and new ones. In this article, we propose an implementation of traffic signs recognition algorithm using a convolution neural network. The paper also shows several CNN architectures, which are compared to each other. Training of the neural network is implemented using the TensorFlow library and massively parallel architecture for multithreaded programming CUDA. The entire procedure for traffic sign detection and recognition is executed in real time on a mobile GPU. The experimental results confirmed high efficiency of the developed computer vision system.

Low-Power Automatic Speech Recognition Through a Mobile GPU and a Viterbi Accelerator

Automatic speech recognition (ASR) has become a core technology for mobile devices. Delivering real-time and accurate ASR has a huge computational cost, which is challenging to achieve in tightly energy-constrained platforms such as mobile devices. A state-of-the-art ASR pipeline consists of a deep neural network (DNN) that converts the audio signal into phonemes' probabilities, followed by a Viterbi search that uses these probabilities to generate a sequence of words. In this article, the authors propose an ASR system for low-power devices that combines a mobile GPU for the DNN with a dedicated hardware accelerator for the Viterbi search. DNN evaluation is easy to parallelize and, hence, it achieves high energy efficiency on a mobile GPU. On the other hand, the Viterbi search is difficult to parallelize, and it represents the main bottleneck for ASR, so the authors propose a hardware accelerator to dramatically reduce its energy requirements while increasing performance. Their proposal outperforms traditional solutions running on the CPU by orders of magnitude. Compared to a GPU-only system, their hybrid scheme combining the GPU and the accelerator improves performance by 5.25 times, while reducing energy by 2.05 times.

Real-time indoor assistive localization with mobile omnidirectional vision and cloud GPU acceleration

In this paper we propose a real-time assistive localization approach to help blind and visually impaired people in navigating an indoor environment. The system consists of a mobile vision front end with a portable panoramic lens mounted on a smart phone, and a remote image feature-based database of the scene on a GPU-enabled server. Compact and e ective omnidirectional image features are extracted and represented in the smart phone front end, and then transmitted to the server in the cloud. These features of a short video clip are used to search the database of the indoor environment via image-based indexing to find the location of the current view within the database, which is associated with floor plans of the environment. A median-filter-based multi-frame aggregation strategy is used for single path modeling, and a 2D multi-frame aggregation strategy based on the candidates’ distribution densities is used for multi-path environmental modeling to provide a final location estimation. To deal with the high computational cost in searching a large database for a realistic navigation application, data parallelism and task parallelism properties are identified in the database indexing process, and computation is accelerated by using multi-core CPUs and GPUs. User-friendly HCI particularly for the visually impaired is designed and implemented on an iPhone, which also supports system configurations and scene modeling for new environments. Experiments on a database of an eight-floor building are carried out to demonstrate the capacity of the proposed system, with real-time response (14 fps) and robust localization results.

L-Bench: An Android benchmark set for low-power mobile GPUs

In recent years GPUs have become one of the most important components in mobile application processors (APs). Thus, performance measurement and analysis of mobile GPUs are crucial to mobile AP manufacturers, device manufacturers, graphics application programmers, and end users. However, it is hard to analyze mobile GPUs in depth via existing high-level (with frames per second) or low-level benchmarks (with a fill rate, ALU performance, etc.). To bridge the gap between the benchmarks, we present a novel Android benchmark set for low-power GPUs, called L-Bench. This benchmark set consists of mid-level micro-benchmarks implemented on OpenGL ES 3.1, which are carefully chosen for different workloads. By analyzing the results, this benchmark suite provides not only frames per second of each benchmark but also performance of each GPU subsystem (geometry units, ALUs, texture mapping units, raster operations pipelines, caches/memory units, and tessellators) and overall GPU performance. For experiments, we perform our benchmark suite on five representative mobile devices that have different mobile GPUs, after that, we describe comprehensive analysis of each GPU architecture.

Interactive 3D object recognition pipeline on mobile GPGPU computing platforms using low-cost RGB-D sensors

In this work, we propose the implementation of a 3D object recognition system which will be optimized to operate under demanding time constraints. The system must be robust so that objects can be recognized properly in poor light conditions and cluttered scenes with significant levels of occlusion. An important requirement must be met: The system must exhibit a reasonable performance running on a low power consumption mobile GPU computing platform (NVIDIA Jetson TK1) so that it can be integrated in mobile robotics systems, ambient intelligence or ambient-assisted living applications. The acquisition system is based on the use of color and depth (RGB-D) data streams provided by low-cost 3D sensors like Microsoft Kinect or PrimeSense Carmine. The resulting system is able to recognize objects in a scene in less than 7 seconds, offering an interactive frame rate and thus allowing its deployment on a mobile robotic platform. Because of that, the system has many possible applications, ranging from mobile robot navigation and semantic scene labeling to human–computer interaction systems based on visual information. A video showing the proposed system while performing online object recognition in various scenes is available on our project website ( http://www.dtic.ua.es/~agarcia/3dobjrecog-jetsontk1/ ).

Adaptive transfer functions

Medical datasets are continuously increasing in size. Although larger models may be available for certain research purposes, in the common clinical practice the models are usually of up to $$512 \times 512 \times 2000$$512×512×2000 voxels. These resolutions exceed the capabilities of conventional GPUs, the ones usually found in the medical doctors' desktop PCs. Commercial solutions typically reduce the data by downsampling the dataset iteratively until it fits the available target specifications. The data loss reduces the visualization quality and this is not commonly compensated with other actions that might alleviate its effects. In this paper, we propose adaptive transfer functions, an algorithm that improves the transfer function in downsampled multiresolution models so that the quality of renderings is highly improved. The technique is simple and lightweight, and it is suitable, not only to visualize huge models that would not fit in a GPU, but also to render not-so-large models in mobile GPUs, which are less capable than their desktop counterparts. Moreover, it can also be used to accelerate rendering frame rates using lower levels of the multiresolution hierarchy while still maintaining high-quality results in a focus and context approach. We also show an evaluation of these results based on perceptual metrics.

Mobile GPU Computing Based Filter Bank Convolution for Three-Dimensional Wavelet Transform

Mobile GPU computing, or System on Chip with embedded GPU (SoC GPU), becomes in great demand recently. Since these SoCs are designed for mobile devices with real-time applications such as image processing and video processing, high-efficient implementations of wavelet transform are essential for these chips. In this paper, the author develops two SoC GPU based DWT: signal based parallelization for discrete wavelet transform (sDWT) and coefficient based parallelization for discrete wavelet transform (cDWT), and the author evaluates the performance of three-dimensional wavelet transform on SoC GPU Tegra K1. Computational results show that, SoC GPU based DWT is significantly faster than SoC CPU based DWT. Computational results also show that, sDWT can generally satisfy the requirement of real-time processing (30 frames per second) with the image sizes of 352×288, 480×320, 720×480 and 1280×720, while cDWT can only obtain read-time processing with small image sizes of 352×288 and 480×320.

Compressed Coverage Masks for Path Rendering on Mobile GPUs.

We present an algorithm to accelerate resolution independent curve rendering on mobile GPUs. Recent trends in graphics hardware have created a plethora of compressed texture formats specific to GPU manufacturers. However, certain implementations of platform independent path rendering require generating grayscale textures on the CPU containing the extent that each pixel is covered by the curve. In this paper, we demonstrate that generating a compressed grayscale texture prior to uploading it to the GPU creates faster rendering times in addition to the memory savings. We implement a real-time compression technique for coverage masks and compare our results against the GPU-based implementation of the highly optimized Skia rendering library. We also analyze the worst case properties of our compression algorithms. We observe up to a 2 × speed improvement over the existing GPU-based methods in addition to up to a 9:1 improvement in GPU memory gains. We demonstrate the performance on multiple mobile platforms.

Compressed Coverage Masks for Path Rendering on Mobile GPUs

Compressed Coverage Masks for Path Rendering on Mobile GPUs

A 130.3 mW 16-Core Mobile GPU With Power-Aware Pixel Approximation Techniques

Intensive pixel shading dominates the power dissipation of the graphics pipeline as the screen resolution grows. In this work, we propose a 130.3 mW 16-core mobile GPU with three pixel approximation techniques and a corresponding tile-based rasterization architecture. The proposed architecture can trade-off between power consumption and visual quality to provide power-aware capability, and is fabricated with TSMC 45 nm technology. The feasibility and effectiveness of these techniques are verified in this chip prototype. The implementation results show that, with satisfactory visual quality, 52.32% of the power consumption of the shader processors can be empirically reduced with an experimental Approximated Precision Shader architecture and a Screen-space Approximated Lighting technique. Furthermore, the Approximated Texturing technique can reduce 24.57% of L1 cache updates in our evaluation.

Toggle-Aware Compression for GPUs

Memory bandwidth compression can be an effective way to achieve higher system performance and energy efficiency in modern data-intensive applications by exploiting redundancy in data. Prior works studied various data compression techniques to improve both capacity (e.g., of caches and main memory) and bandwidth utilization (e.g., of the on-chip and off-chip interconnects). These works addressed two common shortcomings of compression: (i) compression/decompression overhead in terms of latency, energy, and area, and (ii) hardware complexity to support variable data size. In this paper, we make the new observation that there is another important problem related to data compression in the context of the communication energy efficiency: transferring compressed data leads to a substantial increase in the number of bit toggles (communication channel switchings from 0 to 1 or from 1 to 0). This, in turn, increases the dynamic energy consumed by on-chip and off-chip buses due to more frequent charging and discharging of the wires. Our results, for example, show that the bit toggle count increases by an average of 2.2× with some compression algorithms across 54 mobile GPU applications. We characterize and demonstrate this new problem across a wide variety of 221 GPU applications and six different compression algorithms. To mitigate the problem, we propose two new toggle-aware compression techniques: energy control and Metadata Consolidation. These techniques greatly reduce the bit toggle count impact of the six data compression algorithms we examine, while keeping most of their bandwidth reduction benefits.

Fast filter bank convolution for three-dimensional wavelet transform by shared memory on mobile GPU computing

Mobile GPU applications usually constrain by the real-time requirement. However, FLOPS of mobile GPU is limited by the size and power supply of the SoC systems. Same to desktop GPUs, the mobile GPU...

Accelerating single-image super-resolution polynomial regression in mobile devices

This paper introduces a new super-resolution algorithm based on machine learning along with a novel hybrid implementation for next generation mobile devices. The proposed super-resolution algorithm entails a two dimensional polynomial regression method using only the input image properties for the learning task. Model selection is applied for defining the optimal degree of polynomial by adopting regularization capability in order to avoid overfitting. Although it is widely believed that machine learning algorithms are not appropriate for real-time implementation, the paper in hand proves that there are indeed specific hypothesis representations that are able to be integrated into real-time mobile applications. With aim to achieve this goal, the increasing GPU employment in modern mobile devices is exploited. More precisely, by utilizing the mobile GPU as a co-processor in a hybrid pipelined implementation, significant performance speedup along with superior quantitative results can be achieved. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .