Mobile Graphics Research Articles

Mobile Volume Rendering: Past, Present and Future.

Volume rendering has been a relevant topic in scientific visualization for the last decades. However, the exploration of reasonably big volume datasets requires considerable computing power, which has limited this field to the desktop scenario. But the recent advances in mobile graphics hardware have motivated the research community to overcome these restrictions and to bring volume graphics to these ubiquitous handheld platforms. This survey presents the past and present work on mobile volume rendering, and is meant to serve as an overview and introduction to the field. It proposes a classification of the current efforts and covers aspects such as advantages and issues of the mobile platforms, rendering strategies, performance and user interfaces. The paper ends by highlighting promising research directions to motivate the development of new and interesting mobile volume solutions.

Comparison of OpenCL and RenderScript for mobile devices

With the recent advances in the programmability and performance of mobile Graphics Processing Units (GPUs), General-Purpose Graphics Processing Unit (GPGPU) technologies have become available even ...

The Effect of Motion Graphics in Mobile AR Interface on Self-efficacy and Cognitive Attitude

The rapid development of the mobile device has brought about a change of a new lifestyle to users based on the fusion of various IT technologies. the paradigm of digital contents was simply focused on efficiency of processing and delivery of information, today the focus has shifted toward more immediate, intuitive, real, and natural flow such as virtual reality and augmented reality. Especially the AR (Augmented Reality) naturally connected the virtual object to the real world to provide users with the effective mobile user environment. Therefore, the GUI (Graphic User Interface) of the mobile device-based augmented reality needs to be designed properly to the changed mobile environment. However, although the augmented reality is based on the video of the real world which is treated in real time, the study on the motion graphic of the virtual object organizing the user interface is not enough. Therefore, this study is aimed at empirically analyzing the effect of the motion graphic in the user interface of the mobile AR. Also, the positive effect of characteristics on improvement of Users' visual experience, self-efficacy and cognitive attitude in mobile AR motion graphics was examined. The result showed that augmented reality has a significant effect on the self-efficacy and cognitive attitude of the users' visual experience. In the future, more media research for various motion graphics of AR interface will be needed, and this study can help develop mobile AR contents.

Dynamically Selective Performance Optimization Method for Mobile 3D Graphics Application with N-Screen Service

In this paper, we focus on a dynamically selective performance optimization method for mobile 3D graphics applications under n-screen service environment. Thus, an effective solution to enhance the performance is required to support real time processing under limited network bandwidth. This method can be employed to apply quality loss techniques that trade the quality within the range of tolerance over performance. First, we describe performance optimization methods for mobile graphics applications in 2 dimensions: Graphic Processing Unit (GPU) performance and image quality. Second, we present an adaptive level-of-detail (LOD) configuration method for graphics applications via OpenGL ES API level. Third, we show how OpenGL ES library hooking method can be employed to apply quality loss techniques that trade quality over performance. Last, we argue that OpenGL ES library hooking method is crucial to achieve higher performance, since it can be used in common android phones without any source code modification of applications. Overall, the experimental result shows that the proposed technique allows 24 % of performance improvement for all applied methodologies, and 4 % improvement for some selected methodologies while causing around 14 and 2 % of quality degradation, respectively. Thus, our approach can support an acceptable trade-off by determining the level of reduced quality that would lead to the desired performance improvements of the mobile GPU.

Hybrid radix‐4/‐8 truncated multiplier for mobile GPU applications

A hybrid radix-4/-8 truncated multiplier is proposed for mobile graphics processing unit (GPU) applications by combining the strong points of high-speed radix-4 and low-power radix-8 encoding schemes. A hybrid Booth encoder (HBE) for radix-4/-8 encoding is proposed by sharing the common logics between the two encodings. In addition, a hybrid radix-4/-8 truncation (HRT) scheme is proposed for more power reduction at a reasonable peak signal-to-noise ratio loss for mobile multimedia applications. Experimental results demonstrate 29.7 and 31.1% power savings from the HBE and the HRT schemes, respectively, resulting in a total of 60.7% power reduction from the previous work.

Extending the graphics pipeline with adaptive, multi-rate shading

Due to complex shaders and high-resolution displays (particularly on mobile graphics platforms), fragment shading often dominates the cost of rendering in games. To improve the efficiency of shading on GPUs, we extend the graphics pipeline to natively support techniques that adaptively sample components of the shading function more sparsely than per-pixel rates. We perform an extensive study of the challenges of integrating adaptive, multi-rate shading into the graphics pipeline, and evaluate two- and three-rate implementations that we believe are practical evolutions of modern GPU designs. We design new shading language abstractions that simplify development of shaders for this system, and design adaptive techniques that use these mechanisms to reduce the number of instructions performed during shading by more than a factor of three while maintaining high image quality.

C‐LOD: Context‐aware Material Level‐of‐Detail applied to Mobile Graphics

AbstractAttention‐based Level‐Of–Detail (LOD) managers downgrade the quality of areas that are expected to go unnoticed by an observer to economize on computational resources. The perceptibility of lowered visual fidelity is determined by the accuracy of the attention model that assigns quality levels. Most previous attention based LOD managers do not take into account saliency provoked by context, failing to provide consistently accurate attention predictions. In this work, we extend a recent high level saliency model with four additional components yielding more accurate predictions: an object‐intrinsic factor accounting for canonical form of objects, an object‐context factor for contextual isolation of objects, a feature uniqueness term that accounts for the number of salient features in an image, and a temporal context that generates recurring fixations for objects inconsistent with the context. We conduct a perceptual experiment to acquire the weighting factors to initialize our model. We design C‐LOD, a LOD manager that maintains a constant frame rate on mobile devices by dynamically re‐adjusting material quality on secondary visual features of non‐attended objects. In a proof of concept study we establish that by incorporating C‐LOD, complex effects such as parallax occlusion mapping usually omitted in mobile devices can now be employed, without overloading GPU capability and, at the same time, conserving battery power.

Design and Research on Product Aesthetics Based Human-Computer Interaction Graphical Interface

In order to improve the user experience and aesthetic feeling of interactive interface of mobile products, the paper iphone Mobile graphics and interaction design as the foundation, implements the user interface design with means from visual sense (picture), auditory sense (stereophonic sounds),tactile sense (fingers control on function keys) from aesthetic observation points, develops in aesthetic human-computer interactive method, emphasizes on the interaction among humans to give users more direct, healthy and overall game experience.

3차원 그래픽스 기하 파이프라인 기반의 래스터 파이프라인 구현

래스터 연산은 트루 컬러 이미지(픽스맵)나, 단색 이미지(비트맵)을 표현하기 위해서 광범위하게 사용된다. 이 기능은 이미지 프로세싱 기능이나, 폰트 출력 시에 강하게 요구된다. 반면에, OpenGL ES 하드웨어 등을 포함하는, 현재의 모바일 그래픽스 플랫폼들에서는 이 기능을 직접 제공하지는 않는다. 모바일 그래픽스 플랫폼들에서 이러한 래스터 연산을 완벽히 제공하기 위해서, 본 논문에서는 그래픽스 이미지들을 3차원 점들의 집합으로 해석하고, 풀-소프트웨어 구현 방식으로, 이들 3차원 점들을 전형적인 3차원 기하 파이프라인으로 처리하게 했다. 구현 결과는 충분한 실행 속도를 보였고, 정확도를 증명하기 위한 공식 검증 테스트(conformance test)들을 모두 통과하였다. Raster operations are widely used to display full-color graphics images (or pixmaps) and single-color images (or bitmaps). These features are strongly needed for image processing applications and font output. However, current mobile graphics platforms, including OpenGL ES hardware implementations, do not directly support these features. To fully support those raster operations on the mobile graphics platforms, we interpreted the graphics images as a set of 3D points, and processed those 3D points through the typical 3D geometry pipelines, in a full-software implementation. Our implementation shows sufficient execution speeds, and passed the official conformance tests to show its correctness.

Real-Time Segmentation of Stereo Videos on a Portable System With a Mobile GPU

In mobile robotic applications, visual information needs to be processed fast despite resource limitations of the mobile system. Here, a novel real-time framework for model-free spatiotemporal segmentation of stereo videos is presented. It combines real-time optical flow and stereo with image segmentation and runs on a portable system with an integrated mobile graphics processing unit. The system performs online, automatic, and dense segmentation of stereo videos and serves as a visual front end for preprocessing in mobile robots, providing a condensed representation of the scene that can potentially be utilized in various applications, e.g., object manipulation, manipulation recognition, visual servoing. The method was tested on real-world sequences with arbitrary motions, including videos acquired with a moving camera.

A real-time remote rendering system for interactive mobile graphics

Mobile devices are gradually changing people's computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to desktops. Many applications with intensive graphics rendering workloads are unable to run on mobile platforms directly. This issue can be addressed with the idea of remote rendering: the heavy 3D graphics rendering computation runs on a powerful server and the rendering results are transmitted to the mobile client for display. However, the simple remote rendering solution inevitably suffers from the large interaction latency caused by wireless networks, and is not acceptable for many applications that have very strict latency requirements.In this article, we present an advanced low-latency remote rendering system that assists mobile devices to render interactive 3D graphics in real-time. Our design takes advantage of an image based rendering technique: 3D image warping, to synthesize the mobile display from the depth images generated on the server. The research indicates that the system can successfully reduce the interaction latency while maintaining the high rendering quality by generating multiple depth images at the carefully selected viewpoints. We study the problem of viewpoint selection, propose a real-time reference viewpoint prediction algorithm, and evaluate the algorithm performance with real-device experiments.

Area-power efficient lighting unit architecture based on hardware sharing

A shared lighting unit architecture suitable for mobile 3D graphics is presented. Sharing of resources is achieved by reusing a unitary vector operation, and diffuse and specular components calculation. Balance between area and power dissipation was compared with a more prevalent parallel unit architecture implementation. Results indicate that parallel implementations of the proposed shared units lead to much better area × power efficiency as more of these units are used for higher processing performance.

Imperceptible Simplification on Mobile Displays

Graphics on mobile devices is becoming popular because untethered computing is convenient and makes workers more productive. Mobile displays have a wide range of resolutions that affect the scene Level-of-Detail (LoD) that users can perceive: smaller displays show less detail, therefore lower resolution meshes and textures are acceptable. Mobile devices frequently have limited battery energy, low memory and disk space. To minimize wasting limited system resources, the authors render mobile graphics scenes at the lowest LoD at which users do not perceive distortion due to simplification. This is called LoD the Point of Imperceptibility (PoI). Increasing the mesh or texture resolution beyond the PoI wastes valuable system resources without increasing perceivable visual realism. The authors propose a perceptual metric that can easily be evaluated to identify the LoD corresponding to a target mobile display’s PoI and accounts for object geometry, lighting and shading. Previous work did not directly compute changes in the PoI due to target screen resolution. The perceptual metric generates a screen-dependent Pareto distribution with a knee point that corresponds to the PoI. We employ wavelets for simplification, which gives direct access to the mesh undulation frequency that we then use to parameterize the CSF curve.

A Memory-Efficient Unified Early Z-Test

The Unified Early Z-Test (U-EZT) is proposed to examine the visibility of pixels during tile-based rasterization in a mobile 3D graphics processor. U-EZT combines the advantages of the Z-max and Z-min EZT algorithms: the Z-max algorithm is improved by the independently updatable z-max tiles and the use of mask bits; and the Z-min algorithm is improved by reusing the mask bits from the z-max test to update the z-min tiles after tile rasterizing. As a result, storage requirements are reduced to 3 bits per pixel, and simulations suggest that U-EZT requires 20 percent to 57 percent less memory bandwidth than previous EZT algorithms.

Short mobile game development projects for CS1/2

Game development and mobile computing have been successfully used to increase student motivation. However, instructors with no background in mobile computing, computer graphics, and/or game develop...

Compression of 3D Mesh Geometry and Vertex Attributes for Mobile Graphics

This paper presents a compression scheme for mesh geometry, which is suitable for mobile graphics. The main focus is to enable real-time decoding of compressed vertex positions while providing reasonable compression ratios. Our scheme is based on local quantization of vertex positions with mesh partitioning. To prevent visual seams along the partitioning boundaries, we constrain the locally quantized cells of all mesh partitions to have the same size and aligned local axes. We propose a mesh partitioning algorithm to minimize the size of locally quantized cells, which relates to the distortion of a restored mesh. Vertex coordinates are stored in main memory and transmitted to graphics hardware for rendering in the quantized form, saving memory space and system bus bandwidth. Decoding operation is combined with model geometry transformation, and the only overhead to restore vertex positions is one matrix multiplication for each mesh partition. In our experiments, a 32-bit floating point vertex coordinate is quantized into an 8-bit integer, which is the smallest data size supported in a mobile graphics library. With this setting, the distortions of the restored meshes are comparable to 11-bit global quantization of vertex coordinates. We also apply the proposed approach to compression of vertex attributes, such as vertex normals and texture coordinates, and show that gains similar to vertex geometry can be obtained through local quantization with mesh partitioning.

On Balancing Energy Consumption, Rendering Speed, and Image Quality on Mobile Devices

Mobile games and graphics are popular because un-tethered computing is convenient and ubiquitous entertainment is compelling. However, rendering graphics on mobile devices faces challenges due to limited system resources, such as battery energy, and low memory and disk space. Real time frame rates, low energy consumption and high image quality are all desirable attributes of interactive mobile graphics; however, achieving these objectives is conflicting. For instance, increasing mesh resolutions improves rendered image quality but consumes more battery energy. Therefore, the authors propose a mobile graphics heuristic to minimize energy consumption while maintaining acceptable image quality and interactive frame rates. Over the lifetime of a mobile graphics application, scene complexity, animation paths, user interactivity and other elements all change its CPU and resource demands. In this regard, a heuristic that dynamically changes scene mesh LoDs and amount of CPU timeslices allotted to the mobile graphics application is presented to select optimal operating conditions that balance rendering speed, energy conservation and image quality. Additionally, a workload predict model is proposed so that the heuristic can monitor both application workload and the availability of resources of mobile devices periodically, while adaptively determining how much resources will be allocated to applications.

A 186-Mvertices/s 161-mW Floating-Point Vertex Processor With Optimized Datapath and Vertex Caches

In this paper, a power efficient vertex processor for mobile graphics applications is presented. A four-threaded and four-issue expanded VLIW datapath with a quad-float vertex texture fetcher is proposed by exploiting graphics specific characteristics after evaluation of several candidate architectures. Instruction-level power control methods such as operand sharing and writeback re-allocation along with operand isolations and gated clocks result in 40.4% and 82% reduction in energy dissipation and energy delay product compared to the most widely used single threaded SIMD. The proposed processor with the optimized datapath and vertex caches implemented in a 0.18- mum 1P4M CMOS process achieves 186-Mvertices/s geometry performance which is the best result among the processors that are IEEE-754 compliant.

Memory bandwidth saving by hardware tessellation with vertex shader

Hardware architecture of a tessellator based on a vertex shader is proposed to save memory bandwidth for a mobile 3D graphics engine. According to the appropriate separation of the tessellation process, the powerful performance of a vertex shader, 120 Mvertices/s, is utilised for floating point computations. The remaining part of the process is handled by a dedicated control unit, implementation of which requires 6.2% additional logic gates. The proposed tessellator reduces the bandwidth consumed to transfer 3D geometry data up to 1/250.

The State of the Art in Mobile Graphics Research

High-quality computer graphics let mobile-device users access more compelling content. Still, the devices' limitations and requirements differ substantially from those of a PC. This survey of mobile graphics research describes current solutions in terms of specialized hardware (including 3D displays), rendering and transmission, visualization, and user interfaces.