Real-time Graphics Research Articles

Feature-Based Volumetric Terrain Generation and Decoration.

Two-dimensional height fields are the most common data structure used for storing and rendering of terrain in offline rendering and especially real-time computer graphics. By its very nature, a height field cannot store terrain structures with multiple vertical layers such as overhanging cliffs, caves, or arches. This restriction does not apply to volumetric data structures. However, the workflow of manual modelling and editing of volumetric terrain usually is tedious and very time-consuming. Therefore, we propose to use three-dimensional curve-based primitives to efficiently model prominent, large-scale terrain features. We present a technique for volumetric generation of a complete terrain surface from the sparse input data by means of diffusion-based algorithms. By combining an efficient, feature-based toolset with a volumetric terrain representation, the modelling workflow is accelerated and simplified while retaining the full artistic freedom of volumetric terrains. Feature Curves also contain material information that can be complemented with local details by using per-face texture mapping. All stages of our method are GPU-accelerated using compute shaders to ensure interactive editing of terrain. Please note that this paper is an extended version of our previously published work[1] .

Static stages for heterogeneous programming

Heterogeneous hardware is central to modern advances in performance and efficiency. Mainstream programming models for heterogeneous architectures, however, sacrifice safety and expressiveness in favor of low-level control over performance details. The interfaces between hardware units consist of verbose, unsafe APIs; hardware-specific languages make it difficult to move code between units; and brittle preprocessor macros complicate the task of specializing general code for efficient accelerated execution. We propose a unified low-level programming model for heterogeneous systems that offers control over performance, safe communication constructs, cross-device code portability, and hygienic metaprogramming for specialization. The language extends constructs from multi-stage programming to separate code for different hardware units, to communicate between them, and to express compile-time code optimization. We introduce static staging, a different take on multi-stage programming that lets the compiler generate all code and communication constructs ahead of time. To demonstrate our approach, we use static staging to implement BraidGL, a real-time graphics programming language for CPU-GPU systems. Current real-time graphics software in OpenGL uses stringly-typed APIs for communication and unsafe preprocessing to generate specialized GPU code variants. In BraidGL, programmers instead write hybrid CPU-GPU software in a unified language. The compiler statically generates target-specific code and guarantees safe communication between the CPU and the graphics pipeline stages. Example scenes demonstrate the language's productivity advantages: BraidGL eliminates the safety and expressiveness pitfalls of OpenGL and makes common specialization techniques easy to apply. The case study demonstrates how static staging can express core placement and specialization in general heterogeneous programming.

アドリブシステムによるリアルタイムCGキャラクタ制御(マルチメディア(システム/通信/ネットワーク),放送通信連携サービスとその品質,一般)

アドリブシステムによるリアルタイムCGキャラクタ制御(マルチメディア(システム/通信/ネットワーク),放送通信連携サービスとその品質,一般)

Resolution Independent Real-Time Vector-Embedded Mesh for Animation

High-resolution textures are determinant of not only high rendering quality in gaming and movie industries, but also of burdens in memory usage, data transmission bandwidth, and rendering efficiency. Therefore, it is desirable to shade 3D objects with vector images such as scalable vector graphics (SVG) for compactness and resolution independence. However, complicated geometry and high rendering cost limit the rendering effectiveness and efficiency of vector texturing techniques. In order to overcome these limitations, this paper proposes a real-time resolution-independent vector-embedded shading method for 3D animated objects. Our system first decomposes a vector image consisting of layered close coloring regions into unifying-coloring units for mesh retriangulation and 1D coloring texture construction, where coloring denotes color determination for a point based on an intermediate medium such as a raster/vector image, unifying denotes the usage of the same set of operations, and unifying coloring denotes coloring with the same-color computation operations. We then embed the coloring information and distances to enclosed unit boundaries in retriangulated vertices to minimize embedded information, localize vertex-embedded shading data, remove overdrawing inefficiency, and ensure fixed-length shading instructions for data compactness and avoidance of indirect memory accessing and complex programming structures when using other shading and texturing schemes. Furthermore, stroking is the process of laying down a fixed-width pen-centered element along connected curves, and our system also decomposes these curves into segments using their curve-mesh intersections and embeds their control vertices as well as their widths in the intersected triangles to avoid expensive distance computation. Overall, our algorithm enables high-quality real-time Graphics Processing Unit (GPU)-based coloring for real-time 3D animation rendering through our efficient SVG-embedded rendering pipeline while using a small amount of texture memory and transmission bandwidth.

Shader components

Modern game engines seek to balance the conflicting goals of high rendering performance and productive software development. To improve CPU performance, the most recent generation of real-time graphics APIs provide new primitives for performing efficient batch updates to shader parameters. However, modern game engines featuring large shader codebases have struggled to take advantage of these benefits. The problem is that even though shader parameters can be organized into efficient modules bound to the pipeline at various frequencies, modern shading languages lack corresponding primitives to organize shader logic (requiring these parameters) into modules as well. The result is that complex shaders are typically compiled to use a monolithic block of parameters, defeating the design, and performance benefits, of the new parameter binding API. In this paper we propose to resolve this mismatch by introducing shader components , a first-class unit of modularity in a shader program that encapsulates a unit of shader logic and the parameters that must be bound when that logic is in use. We show that by building sophisticated shaders out of components, we can retain essential aspects of performance (static specialization of the shader logic in use and efficient update of parameters at component granularity) while maintaining the modular shader code structure that is desirable in today's high-end game engines.

Texture Stationarization: Turning Photos into Tileable Textures

Texture synthesis has grown into a mature field in computer graphics, allowing the synthesis of naturalistic textures and images from photographic exemplars. Surprisingly little work, however, has been dedicated to synthesizing tileable textures, that is, textures that when laid out in a regular grid of tiles form a homogeneous appearance suitable for use in memory-sensitive real-time graphics applications. One of the key challenges in doing so is that most natural input exemplars exhibit uneven spatial variations that, when tiled, show as repetitive patterns. We propose an approach to synthesize tileable textures while enforcing stationarity properties that effectively mask repetitions while maintaining the unique characteristics of the exemplar. We explore a number of alternative measures for texture stationarity and show how each measure can be integrated into a standard texture synthesis method PatchMatch to enforce stationarity at user-controlled scales. We demonstrate the efficacy of our approach using a database of 118 exemplar images, both from publicly available sources as well as new ones captured under uncontrolled conditions, and we quantitatively analyze alternative stationarity measures for their robustness across many test runs using different random seeds. In conclusion, we suggest a novel synthesis approach that employs local histogram matching to reliably turn input photographs of natural surfaces into tiles well suited for artifact-free tiling.

Research and Application of Collaborative Processing Technology for Graphics and Image Based on HTML5

Based on the above defects and the superiority of HTMLS, the paper is aimed at realizing the process of graphics and image on Web by combining with graphics and image processing algorithm on computer at present. At the same time, the real-time graphics and image processing cooperatively by multiple personnel is also realized. Finally, a graphics and image processing platform on Web will be established which is easy to maintain, share, expand, being carried out in real time and iteratively. It is named as Web Photoshop. The platform is not only able to carry out graphics and image processing work on a single computer, but also used for graphics and image processing cooperatively by multiple personnel. At the same time, the subject also analyzes and discusses the possibility of graphics and image processing cooperatively by multiple personnel by comparing the effect of common graphics and image processing program on desktop. What is more, the HTMLS is used in trial in the subject to rapidly construct the graphics and image processing application based on mobile devices.

VennDiagramWeb: a web application for the generation of highly customizable Venn and Euler diagrams.

BackgroundVisualization of data generated by high-throughput, high-dimensionality experiments is rapidly becoming a rate-limiting step in computational biology. There is an ongoing need to quickly develop high-quality visualizations that can be easily customized or incorporated into automated pipelines. This often requires an interface for manual plot modification, rapid cycles of tweaking visualization parameters, and the generation of graphics code. To facilitate this process for the generation of highly-customizable, high-resolution Venn and Euler diagrams, we introduce VennDiagramWeb: a web application for the widely used VennDiagram R package. VennDiagramWeb is hosted at http://venndiagram.res.oicr.on.ca/.ResultsVennDiagramWeb allows real-time modification of Venn and Euler diagrams, with parameter setting through a web interface and immediate visualization of results. It allows customization of essentially all aspects of figures, but also supports integration into computational pipelines via download of R code. Users can upload data and download figures in a range of formats, and there is exhaustive support documentation.ConclusionsVennDiagramWeb allows the easy creation of Venn and Euler diagrams for computational biologists, and indeed many other fields. Its ability to support real-time graphics changes that are linked to downloadable code that can be integrated into automated pipelines will greatly facilitate the improved visualization of complex datasets. For application support please contact Paul.Boutros@oicr.on.ca.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-016-1281-5) contains supplementary material, which is available to authorized users.

Discrete Method of Images for 3D Radio Propagation Modeling

Discretization by rasterization is introduced into the method of images (MI) in the context of 3D deterministic radio propagation modeling as a way to exploit spatial coherence of electromagnetic propagation for fine-grained parallelism. Traditional algebraic treatment of bounding regions and surfaces is replaced by computer graphics rendering of 3D reflections and double refractions while building the image tree. The visibility of reception points and surfaces is also resolved by shader programs. The proposed rasterization is shown to be of comparable run time to that of the fundamentally parallel shooting and bouncing rays. The rasterization does not affect the signal evaluation backtracking step, thus preserving its advantage over the brute force ray-tracing methods in terms of accuracy. Moreover, the rendering resolution may be scaled back for a given level of scenario detail with only marginal impact on the image tree size. This allows selection of scene optimized execution parameters for faster execution, giving the method a competitive edge. The proposed variant of MI can be run on any GPU that supports real-time 3D graphics.

A system for rapid exploration of shader optimization choices

We present Spire, a shading language and compiler framework that facilitates rapid exploration of shader optimization choices (such as frequency reduction and algorithmic approximation) afforded by modern real-time graphics engines. Our design combines ideas from rate-based shader programming with new language features that expand the scope of shader execution beyond traditional GPU hardware pipelines, and enable a diverse set of shader optimizations to be described by a single mechanism: overloading shader terms at various spatio-temporal computation rates provided by the pipeline. In contrast to prior work, neither the shading language's design, nor our compiler framework's implementation, is specific to the capabilities of any one rendering pipeline, thus Spire establishes architectural separation between the shading system and the implementation of modern rendering engines (allowing different rendering pipelines to utilize its services). We demonstrate use of Spire to author complex shaders that are portable across different rendering pipelines and to rapidly explore shader optimization decisions that span multiple compute and graphics passes and even offline asset preprocessing. We further demonstrate the utility of Spire by developing a shader level-of-detail library and shader auto-tuning system on top of its abstractions, and demonstrate rapid, automatic re-optimization of shaders for different target hardware platforms.

The Virtual Reality Art Installation Endocytosis: Evolving from a Flat Land into a Three-Dimensional World

Virtual reality systems are an ideal platform for exploring spatial effects because of their ability to combine stereo imaging techniques and interactive real-time graphics. They allow the creation of artworks that, on the one hand, exhibit a dynamic organization of the environment’s spatial depth and, on the other, create an interaction with the stereoscopic optic flow. In this article, the author discusses the advantages of horizontal stereoscopic displays and describes how she has used the Responsive Workbench to display the evolution of a flat land into a three-dimensional world in her artwork Endocytosis. She uses endocytosis—a fundamental cellular trafficking process that moves material into the intracellular space—as a metaphor for this evolutionary process.

Memory Access Scheduling for a Smart TV

A smart TV system-on-chip (SoC) has very heavy computation and memory demands that must be met with low-cost components. As a result, there is potentially an extremely high utilization of the channel between the SoC and its memory chip. This paper presents the design of a new memory access scheduler customized for the type of memory traffic typically encountered with smart TVs. This includes special accumulated hard real-time graphics requirements, user response-sensitive soft real-time requirements, and the need to provide high memory throughput and priority-handling capabilities even under extremely heavy memory traffic conditions. The simulation results show that the proposed memory access scheduler is able to achieve up to 98% of the ideal upper bound memory throughput when faced with extremely heavy memory traffic—this is a significant improvement over previous schedulers. Novel future prediction and light-handed priority handling methods are used to achieve these results while satisfying the unique real-time requirements of smart TVs.

In-Class Exercises for Shadow Volume Algorithms

This article describes a simple course exercise to deepen the understanding of real-time shadow volume algorithms. The exercise takes less than 10 minutes to perform during lecture time, leads to a profounder understanding of the topic, and serves as a basis for further discussions of improvements for the shadow mapping algorithm. Using this exercise, the authors noticed an increase in students implementing shadow algorithms for later real-time graphics projects.

In-class exercise for shadow mapping algorithms.

This article describes a simple course exercise to deepen the understanding of real-time shadow mapping algorithms. The exercise takes less than 10 minutes to perform during lecture time, leads to a profounder understanding of the topic, and serves as a basis for further discussions of improvements for the shadow mapping algorithm. Using this exercise, the authors noticed an increase in implementing shadow algorithms for later real-time graphics projects.

Real-time Ray-tracing Chip Architecture

In this paper, we describe the world’s first real-time ray-tracing chip architecture. Raytracing technology generates high-quality 3D graphics images better than current rasterization technology by providing four essential light effects: shadow, reflection, refraction and transmission. The real-time ray-tracing chip named RayChip includes a real-time ray-tracing graphics processing unit and an accelerating tree-building unit. An ARM Ltd. central processing unit (CPU) and other peripherals are also included to support all processes of 3D graphics applications. Using the accelerating tree-building unit named RayTree to minimize the CPU load, the chip uses a low-end CPU and decreases both silicon area and power consumption. The evaluation results with RayChip show appropriate performance to support real-time ray tracing in high-definition (HD) resolution, while the rendered images are scaled to full HD resolution. The chip also integrates the Linux operating system and the familiar OpenGL for Embedded Systems application programming interface for easy application development.

CellVIEW: a Tool for Illustrative and Multi-Scale Rendering of Large Biomolecular Datasets.

In this article we introduce cellVIEW, a new system to interactively visualize large biomolecular datasets on the atomic level. Our tool is unique and has been specifically designed to match the ambitions of our domain experts to model and interactively visualize structures comprised of several billions atom. The cellVIEW system integrates acceleration techniques to allow for real-time graphics performance of 60 Hz display rate on datasets representing large viruses and bacterial organisms. Inspired by the work of scientific illustrators, we propose a level-of-detail scheme which purpose is two-fold: accelerating the rendering and reducing visual clutter. The main part of our datasets is made out of macromolecules, but it also comprises nucleic acids strands which are stored as sets of control points. For that specific case, we extend our rendering method to support the dynamic generation of DNA strands directly on the GPU. It is noteworthy that our tool has been directly implemented inside a game engine. We chose to rely on a third party engine to reduce software development work-load and to make bleeding-edge graphics techniques more accessible to the end-users. To our knowledge cellVIEW is the only suitable solution for interactive visualization of large bimolecular landscapes on the atomic level and is freely available to use and extend.

A Study of Speed of the Boundary Element Method as applied to the Realtime Computational Simulation of Biological Organs

In this work, possibility of simulating biological organs in realtime using the Boundary Element Method (BEM) is investigated. Biological organs are assumed to follow linear elastostatic material behavior, and constant boundary element is the element type used. First, a Graphics Processing Unit (GPU) is used to speed up the BEM computations to achieve the realtime performance. Next, instead of the GPU, a computer cluster is used. Results indicate that BEM is fast enough to provide for realtime graphics if biological organs are assumed to follow linear elastostatic material behavior. Although the present work does not conduct any simulation using nonlinear material models, results from using the linear elastostatic material model imply that it would be difficult to obtain realtime performance if highly nonlinear material models that properly characterize biological organs are used. Although the use of BEM for the simulation of biological organs is not new, the results presented in the present study are not found elsewhere in the literature.

Efficient Screen Space Anisotropic Blurred Soft Shadows

Shadow mapping is an efficient method to generate shadows in real time computer graphics and has broad variations from hard to soft shadow synthesis. Soft shadowing based on shadow mapping is a blurring technique on a shadow map or on screen space. Blurring on screen space has an advantage for efficient sampling on a shadow map, since the blurred target array has exactly the same coordinates as the screen. However, a previous blurring method on screen space has a drawback: the generated shadow is not correct when a view direction has a large angle to the normal of the shadowed plane. In this paper, we introduce a new screen space based method for soft shadowing that is fast and generates soft shadows more accurately than the previous screen space soft shadow mapping method. The resultant images show shadows produced by our method just stand in the same place, while shadows by the previous method change in terms of penumbra while the view moves. Surprisingly, although our method is more complex than the previous method, the measurement results of the calculation time show our method is almost the same performance. This is because it controls the blurring area more accurately and thus successfully reduces multiplications for blurring.

A Virtual Reality System for Training Operators

This paper presents an immersive system for virtual reality-based training of operators. Recent studies show that virtual reality simulations may be helpful in different fields, namely, industry and education. In this particular case we present a simulator for the training of tower cranes operators reducing their costs and risks. This immersive system integrates real-time 3D computer graphics with other modalities, such as, user pose tracking, stereoscopic head-mounted display, pendant remote controller, physics behaviour, and spatial sound, in order to create a more realistic and immersive experience to the user. The results demonstrate two different simulator setups, one fixed and one portable, where the fixed offer a larger traceable area and the portable can be moved wherever is needed. The simulator is robust, reliable and provides an experience very similar to reality. Furthermore, it is possible to extend the simulation to other types of cranes and different types of scenarios.

The perceptual world of a virtual Umwelt

Abstract Real-time computer graphics and complex sensory input challenge past assumptions of highly constrained metaphors based on static imagery. Access to research and gaming interfaces have popularized the understanding of tracking technologies that tailor interaction to ambulatory displacement and dexterous handling of objects, expanding the realm of metaphors from visual to physical phenomena. But behaviour and the mind have been studied far before there were real-time computer graphics or digitally created synthetic environments. Dynamic relationships between environment, body and thought are being dissected today by neuro-scientific research, and, conversely, synthetic environments are being utilized to address psychological disorders. This article examines research on scientific and philosophical fields that advance the definition of representation, from an image-based realm as limited by previous popular media to an understanding of action and immersion, where attention is activated by change, and rooted in the body. From this perspective, the advent of embodied representation may not be as much of a change of medium, but the perceptual relation with the medium, the medium being a created Umwelt, where the virtual world builds the mind through the body.