Mobile Graphics Research Articles

A unified model and method for forecasting energy consumption in distributed computing systems based on stationary and mobile devices

The subject of research in this article is the forecasting of energy consumption when computing distributed tasks on computer networks built on the basis of server solutions and distributed systems based on personal smartphones. The goal of this study was to create a universal computing energy cost prediction model that can be applied to both traditional and mobile cloud systems. Tasks: conduct an analysis of energy-saving approaches and technologies used to calculate data; consider computer system models and actions with them, namely: model of distributed job, model of distributed computing system, model of distribution strategy; develop a common and uniform dynamic method of forecasting spent energy with a focus on heterogeneous systems; conduct a study of the proposed approach on stationary and mobile devices. The obtained results include. The results of the experimental measurement of the energy consumption of mobile digital systems and stationary ones are presented. The energy efficiency of computing on GPUs of a stationary device based on CUDA technology and GPUs on mobile devices based on Apple Metal technology was determined. Computation during the calculation of 600 frames on a distributed system from mobile devices with failure settings showed a consumption of 15320 joules of energy. Simulation of computing on a distributed system with stationary devices showed a consumption of 52806 joules of energy. This gives us 3,45 times the consumption benefit from computing on mobile devices. Forecasted consumption is also very accurate. Conclusions. The energy consumption assessment model proved to be quite effective. The results of the experiments show that the energy consumption estimation model takes into account the features of the hardware platform where data processing is performed. Computation of data on the GPU of stationary devices loses energy efficiency to a similar implementation on the GPU of Apple Metal from mobile devices. Therefore, the presented results demonstrate the rationality of using mobile graphics processors for energy-efficient information processing.

Texture Inspection using the Voice Based Biometric Authentication in the Quantum Computation

Texture inspection has remained an important substance throughout the whole history because with the advances of the mobile graphics and many hardware, high-quality image processing for the basic wood texture in the Wolfram mathematical. We also used wood in the building material and household furniture to find out the type of wood that can be used as a specific material and to help in the development of the different wood texture in the android studio. Cedar is a fibrous and a porous structural tissue found in the roots of trees and other woody plants. The colour of the wood has been depend on the reflection of the most important component of the wood grain may be the process of refining or protection of the wooden surface in the production of the manufacturing cost 5 and 30%. Various Woods textures having the appearance of the wood that require the wood joints for the great variation between sapwood and heartwood.

Wood Texture Inspection using Android Studio

Wood texture has remained an important substance throughout the whole history because with the advances of the mobile graphics and many hardware, high-quality image processing for the basic wood texture in the android studio. We also used wood in the building material and household furniture to find out the type of wood that can be used as a specific material and to help in the development of the different wood texture in the android studio.

Sustainable AI Processing at the Edge

Edge computing is a popular paradigm for accelerating light- to medium-weight machine learning algorithms initiated from mobile devices without requiring the long communication latencies to send them to remote datacenters in the cloud. Edge servers primarily consider traditional concerns, such as size, weight, and power constraints for their installations. However, such metrics are not entirely sufficient to consider environmental impacts from computing given the significant contributions from embodied energy and carbon. In this article we explore the tradeoffs of hardware strategies for convolutional neural network acceleration engines considering inference and online training. In particular, we explore the use of mobile graphics processing unit (GPU) accelerators, recently released edge-class field-programmable gate arrays, and novel processing in memory (PIM) using dynamic random-access memory (DRAM) and emerging Racetrack memory. Given edge servers already employ DRAM and sometimes GPU accelerators, we consider the sustainability implications using breakeven analysis of replacing or augmenting DDR3 with Racetrack memory. We also consider the implications for provisioning edge servers with different accelerators using indifference analysis. While mobile GPUs are typically much more energy efficient, their significant embodied energy can make them less sustainable than PIM solutions in certain scenarios that consider activity time and compute effort.

Efficient Depth Map Creation with a Lightweight Deep Neural Network

Finding depth information with stereo matching using a deep learning algorithm for embedded systems has recently gained significant attention owing to emerging high-performance mobile graphics processing units (GPUs). Several researchers have proposed feasible small-scale CNNs that can run on a local GPU, but they still suffer from low accuracy and/or high computational requirements. In the method proposed in this study, pooling layers with padding and an asymmetric convolution filter are used to reduce computational costs and simultaneously maintain the accuracy of disparity. The patch size and number of layers are adjusted by analyzing the feature and activation maps. The proposed method forms a small-scale network algorithm suitable for a vision system at the edge and still exhibits high-disparity accuracy and low computational loads as compared to existing stereo-matching networks.

Realistic Lighting for Interactive Applications Using Semi-Dynamic Light Maps

Light mapping is a technique that works by precomputing the lighting of a scene to speed up expensive lighting calculations at run-time. It is a commonly employed technique for computer games, in particular mobile games, due to the difficulty of creating realistic lighting effects on low power devices. The primary drawback of this technique is that the scene state that is dependent on the precomputed data cannot be changed at run-time. This limitation of light mapping significantly decreases the interactability of applications that use light maps. In this paper, we present a method to remove some of these restrictions at the cost of additional texture memory and small CPU/GPU workload. This allows changing the color and intensity properties of selected light sources at run-time, while keeping the benefits of the light mapping technique. It also makes it possible to show or hide the selected objects without giving rise to shadow artifacts and illumination inconsistencies. Our algorithm computes the light maps separately for each light source. Regions shadowed by each semi-dynamic object are also captured and stored. These maps are later combined at run-time to correctly illuminate the scene. Despite the increase in the generation time of the precomputed data, the overhead of the method is low enough to make it useful in many real-time applications. This makes our method especially suitable for interactive computer games designed to run on mobile and low-end graphics processors.

Ambiguity-Free Optical-Inertial Tracking for Augmented Reality Headsets.

The increasing capability of computing power and mobile graphics has made possible the release of self-contained augmented reality (AR) headsets featuring efficient head-anchored tracking solutions. Ego motion estimation based on well-established infrared tracking of markers ensures sufficient accuracy and robustness. Unfortunately, wearable visible-light stereo cameras with short baseline and operating under uncontrolled lighting conditions suffer from tracking failures and ambiguities in pose estimation. To improve the accuracy of optical self-tracking and its resiliency to marker occlusions, degraded camera calibrations, and inconsistent lighting, in this work we propose a sensor fusion approach based on Kalman filtering that integrates optical tracking data with inertial tracking data when computing motion correlation. In order to measure improvements in AR overlay accuracy, experiments are performed with a custom-made AR headset designed for supporting complex manual tasks performed under direct vision. Experimental results show that the proposed solution improves the head-mounted display (HMD) tracking accuracy by one third and improves the robustness by also capturing the orientation of the target scene when some of the markers are occluded and when the optical tracking yields unstable and/or ambiguous results due to the limitations of using head-anchored stereo tracking cameras under uncontrollable lighting conditions.

Regional multiscale Markov random field for remote sensing image classification

According to public and industrial interests on mobile graphics, a preliminary implementation regarding 3D feature visualization system on cell phone was performed using M3G API, one of the de-facto standards for mobile 3D graphic API. Through this experiment, it is revealed that scene graph structure and 3D mobile file format supported from this API is useful one for 3D geo-modeling and rendering in mobile environment. It is necessary that 3D mobile graphic standards can be considered as one component of current mobile GIS services standards to provide value-added 3D GIS contents.

Learning Set Representations for LWIR In-Scene Atmospheric Compensation

Atmospheric compensation of long-wave infrared (LWIR) hyperspectral imagery is investigated in this article using set representations learned by a neural network. This approach relies on synthetic at-sensor radiance data derived from collected radiosondes and a diverse database of measured emissivity spectra sampled at a range of surface temperatures. The network loss function relies on LWIR radiative transfer equations to update model parameters. Atmospheric predictions are made on a set of diverse pixels extracted from the scene, without knowledge of blackbody pixels or pixel temperatures. The network architecture utilizes permutation-invariant layers to predict a set representation, similar to the work performed in point cloud classification. When applied to collected hyperspectral image data, this method shows comparable performance to Fast Line-of-Sight Atmospheric Analysis of Hypercubes-Infrared (FLAASH-IR), using an automated pixel selection approach. Additionally, inference time is significantly reduced compared to FLAASH-IR with predictions made on average in 0.24 s on a 128 pixel by 5000 pixel data cube using a mobile graphics card. This computational speed-up on a low-power platform results in an autonomous atmospheric compensation method effective for real-time, onboard use, while only requiring a diversity of materials in the scene.

McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge

The energy efficiency of accelerating hundreds of MB-large deep neural networks (DNNs) in a mobile environment is less than that of a server-class big chip accelerator because of the limited power budget, silicon area, and smaller buffer size of static random access memory associated with mobile systems. To address this challenge and provide powerful computing capability for processing large DNN models in power/resource-limited mobile systems, we propose McDRAM v2, which is a novel in-dynamic random access memory (DRAM) systolic array accelerator architecture. McDRAM v2 makes the best use of large in-DRAM bandwidths for accelerating various DNN applications. It can handle large DNN models without off-chip memory accesses, in a fast and efficient manner, by exposing the large DRAM capacity and large in-DRAM bandwidth directly to an input systolic array of a processing element matrix. Additionally, it maximizes data reuse using a systolic multiply-accumulate (MAC) structure. The proposed architecture maximizes the utilization of large-scale MAC units by judiciously exploiting the DRAM's internal bus and buffer structure. An evaluation of large DNN models in the fields of image classification, natural language processing, and recommendation systems shows that it achieves 1.7 times tera operations per second (TOPS), 3.7 times TOPS/watt, and 8.6 times TOPS/mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> improvements over a state-of-the-art mobile graphics processing unit accelerator, and 4.1 times better energy efficiency over a state-of-the-art server-class accelerator. Moreover, it incurs a minimal overhead, i.e., a 9.7% increase in area, and uses less than 4.4 W of peak operating power.

Методы визуализации протяженных ландшафтов в тренажерно-обучающих системах

One of the tasks that arise during the development of training (simulation) systems is the creation of a subsystem for displaying the earth's surface. Subsystems of this class provide modeling and visualization of the landscape, underlying surface (rivers, roads, forests, etc.), geometric objects that simulate cities or airports and so on. There are two main approaches to the visualization of extended landscapes --- continuous level of detail algorithms that use clustered triangulation and geometry clipmapbased algorithms. The method of generation and visualization of the Earth's surface presented in the work is based on the second approach. It is possible to perform calculations on the GPU using single precision, which allows faster calculations compared to using data types with double precision. In addition, this approach may be used on mobile graphics processors that do not support double precision. The proposed new method of dynamic resource management reduces the occupied video memory, which allows more detailed texture data to be loaded for a larger number of objects simultaneously.

Design of texture cache for embedded mobile graphics processing unit

Design of texture cache for embedded mobile graphics processing unit

Research and Analysis of the Development Approaches in E-Business Mobile Application

With e-Business rapidly spreads out in recent years, there is a significant trend that more and more consumers are using mobile applications in engaging with e-Commerce. In order to reach out to maximum consumers across diverse mobile platforms, how to choose a suitable e-Commerce mobile application development approaches with reliable compatibility and lower cost becomes a challenging solution. This paper starts with describing an overview of various development approaches in the field of e-Commerce mobile applications. The research will focus on examining the performance of e-Business mobile applications in terms of access to mobile features, advanced graphics and compatibility are developed using such frameworks. By analyzing the research data on several platforms, this paper will list the pros and cons in each development approaches under different commercial criteria. Finally, the paper summarizes the contributions and concludes with some possible future e-Commerce mobile development directions.

Real-time visual transmission mechanism of graphics diversity based on mobile 3D graphics matching algorithm

ABSTRACT A visual communication mechanism for the diversity of advertising graphs based on matching algorithm of mobile 3D graphs is proposed, so as to increase the effectiveness of visual communication mechanism for the diversity of advertising graphs. Firstly, a matching algorithm of mobile 3D graph is proposed, and the optimal prediction block in the search scope of reference frame is searched with the current frame block, thus increasing the effectiveness of visual communication mechanism; secondly, optimal matching block is sought is the search scope of reference frame with boundary pixel of sparse image block, thus improving the performance of matching algorithm; and finally, the performance advantage of the proposed algorithm concerning communication precision and communication efficiency is verified.

Towards Real-Time Traffic Sign Recognition via YOLO on a Mobile GPU

Classification of objects in the video stream with the help of deep learning has gained immense popularity nowadays. Considering many systems solving the classification problem, the mobility is often required. This paper proposes an implementation of the YOLO (You Only Look Once) convolutional neural network to solve the problem of classification of traffic signs on the mobile platform NVIDIA Jetson. The main feature of this platform is the availability of mobile graphics processor NVIDIA Tegra, which allows high-performance computing with low power consumption. The implemented algorithm of the YOLO CNN neural network allows solving the problem of the classification of traffic signs in a continuous video stream with decent accuracy and speed, and the NVIDIA Jetson platform provides mobility of the system.

View-Region Optimized Image-Based Scene Simplification

We present a new algorithm for image-based simplification of complex scenes for virtual reality (VR). The algorithm transforms geometrically-detailed environments into a layered quad tile representation that is optimized for a specified viewing region and is renderable on low-power mobile-class VR devices. A novel constrained optimization formulation ensures that the scene can be rendered within a predetermined compute budget, with limits on both primitive count and fill rate. Furthermore, we introduce a new method for texturing from point samples of the original scene geometry that generates high-quality silhouettes without the drawbacks of traditional point splatting. The resulting representation achieves a visual fidelity that was previously impossible on mobile graphics hardware; our algorithm can typically generate a high-quality representation of visually-rich scenes with billions of triangles using just 72k triangles and a single high-resolution texture map (with generally only about 50% more texels than a stereo panorama). The effectiveness of the approach is demonstrated with a set of challenging test cases.

Mobile GPU-based implementation of automatic analysis method for long-term ECG

BackgroundLong-term electrocardiogram (ECG) is one of the important diagnostic assistant approaches in capturing intermittent cardiac arrhythmias. Combination of miniaturized wearable holters and healthcare platforms enable people to have their cardiac condition monitored at home. The high computational burden created by concurrent processing of numerous holter data poses a serious challenge to the healthcare platform. An alternative solution is to shift the analysis tasks from healthcare platforms to the mobile computing devices. However, long-term ECG data processing is quite time consuming due to the limited computation power of the mobile central unit processor (CPU).MethodsThis paper aimed to propose a novel parallel automatic ECG analysis algorithm which exploited the mobile graphics processing unit (GPU) to reduce the response time for processing long-term ECG data. By studying the architecture of the sequential automatic ECG analysis algorithm, we parallelized the time-consuming parts and reorganized the entire pipeline in the parallel algorithm to fully utilize the heterogeneous computing resources of CPU and GPU.ResultsThe experimental results showed that the average executing time of the proposed algorithm on a clinical long-term ECG dataset (duration 23.0 ± 1.0 h per signal) is 1.215 ± 0.140 s, which achieved an average speedup of 5.81 ± 0.39× without compromising analysis accuracy, comparing with the sequential algorithm. Meanwhile, the battery energy consumption of the automatic ECG analysis algorithm was reduced by 64.16%. Excluding energy consumption from data loading, 79.44% of the energy consumption could be saved, which alleviated the problem of limited battery working hours for mobile devices.ConclusionThe reduction of response time and battery energy consumption in ECG analysis not only bring better quality of experience to holter users, but also make it possible to use mobile devices as ECG terminals for healthcare professions such as physicians and health advisers, enabling them to inspect patient ECG recordings onsite efficiently without the need of a high-quality wide-area network environment.

Integrating Interactive Multimedia Objects in Mobile Augmented Reality for Sarawak Tourism

Augmented Reality (AR) is a form of technology that superimpose virtual object or content in a real-world environment to create a mixture of reality. This research studies ARToolKitPlus tracking and developed an Android based AR system which integrate mobile graphics and interaction modules, such as audio, video and 3D objects. The development of multimedia based mobile AR system involved video capturing, image processing, mobile graphics and interactive multimedia features. Android camera Application Programming Interface (API) was implemented for video capturing purposes. In order to perform image processing tasks, an image library was built such as greyscale transformation and thresholding. The graphics module for this system was built using OpenGLES and PowerVR graphics API. Whereas, audio and video API for Android platform were used to build multimedia based audio and video module. As for interaction module, technique of color picking algorithm was applied. Henceforth, a prototype of mobile Sarawak Tourism system was developed to evaluate the multimedia based modules functionalities testing; text, animations, audio, video, 2D/3D object interactions. A preliminary survey was conducted to measure the interaction performance and usability of multimedia based modules AR on Sarawak Tourism application.

A QoS Aware Resource Allocation Strategy for Mobile Graphics Rendering With Cloud Support

With the rapid development of cloud technology, many services have been transferred from local computers to the cloud-based platform, which decreases the amount of computation done on the former. The local computer could thus be developed in the direction of portability and power saving. Graphics processing, apart from providing user interfaces featuring diversified special effects, is also significant in terms of application programs and play interactions. It is exactly on the basis of the concept of graphics processing that cloud-support rendering is developed, which is aimed to improve the graphics efficiency in mobile devices, via the graphics processing units in the cloud-based platform. The cloud-based platform and the mobile devices are usually connected by the Internet; however, as remote rendering might call for greater network bandwidth, its efficiency will be compromised if the network bandwidth is not stable. Given this limitation, this paper sets out to propose a quality-of-service-aware resource allocation strategy for mobile 3D graphics rendering, which is a hybrid rendering technology combining the client-side graphics processing capabilities with the graphics processing units in the cloud-based platform. When network bandwidth is not stable, the technology is able to assess the current network bandwidth, and dynamically configure the rendered frames on the client side and cloud-based platforms. Even when the client side could not access the network, it would still be possible to carry out the drawing through the graphics processing units on the local computer. Three applications are tested in this research: the technology can increase the frame rate by an average of 44.99% when the bandwidth is 10% greater than the minimum limit, by an average of 44.57% when the bandwidth is less than the minimum limit, by an average of 30.86% when the bandwidth is 10% less than the minimum limit, and by an average of 33.74% when the bandwidth is not stable.

Introduction to this Special Issue on Smart Glasses

Abstract The idea of augmented or virtual reality in combination with head mounted display is being discussed already since at least 1968. However, for a long time, this topic was discussed mainly within the academic research area with only limited effect or uptake in the work place. Primary reason for this was the missing availability of robust and affordable hardware as well as the limited mobile graphics capabilities. This has changed recently with the availability of numerous affordable devices in combination with applications from the entertainment and gaming area. This Special Issue on Smart Glasses presents a mix of recent research papers and reports to provide an overview of ongoing research and developments in work place environments. In the remainder of this introductory paper we present an overview of the history of Smart Glasses and their applications over the last decades. We also clarify the term Augmented Reality in this historic context. Then we present a topology of current products as well as their intended application areas. Finally, we introduce the papers of this issue within this context.