Graphics API Research Articles

Constructing a game engine: A proposed game engine architecture course for undergraduate students

This paper presents a structured series of lessons for teaching undergraduate computer science or game development students how to construct a game engine using C++ and standard graphics APIs such as OpenGL or DirectX. Our proposed course content discusses rendering topics, game object management, scene management, and rigid body physics where learners are tasked to incorporate them into a fully functional prototype scene editor. By incorporating actual coding into the course, students can gain a better understanding of low-level engine features and also hone their programming skills. In addition to the learning content, we provide recommended assessment methods. We include discussions about our teaching experience in handling a game engine architecture course, titled GDENG03, which 36 students have passed. Our paper stands out from related works by providing a framework for interested educators on how to write a 3D game engine and how to teach the same in class. Based on our actual delivery of GDENG03, all 36 students achieved high scores of >80% out of 100 which demonstrates the effectiveness of our learning content. We received positive student feedback, such as a better appreciation of game engines, while suggestions for course improvement were raised, such as using simpler terminologies and better workload management. Based on student scores and feedback, we can attribute the success of our students’ learning to our inclusion of practical coding examples and discussions, highlighting their relevance to mainstream game engines. The learning materials and accompanying source code can be accessed in the following link: .

High performance GPU graphics API abstraction layer in C# for real-time graphics

Real-Time rendering is the technique which allows us to have graphical applications in our everyday life, whether it is a 3D game or a tool with graphical user interface. Nowadays graphics rendering is handled by the GPU (Graphics Processing Unit) in our device. There are many layers of abstraction above the programming of GPUs through libraries and graphics engines, though the most low-level way of accessing a GPU in user-mode applications is using a Graphics API. Due to the need of high performance and low-level capabilities usually these APIs are used from C or C++, but we realized the need to utilize these APIs in higher level languages as well.In our approach we're using the .NET C# language for developing multi-platform real-time graphical applications instead of the C or C++ languages. Using the modern .NET environment, we're able to use Graphics APIs for rendering onto common .NET UI Frameworks while consuming all our previously implemented C# libraries and .NET technologies in the same application. To maintain compatibility with multiple platforms we're developing a library system allowing the use different Graphics APIs from the same C# source-code. The library system contains a Graphics API abstraction layer with multiple Graphics API implementations of this layer in C# and a C# to shader language compiler for cross-API shader development in C#.In this paper, we're proposing our considerations for implementing a library to be able to use the Vulkan and OpenGL APIs through a single C# codebase. We provide solutions for multi-platform rendering and dealing with the low-level challenges of using the two deeply different APIs, while maintaining performance capable to do real-time rendering.

A Case for Graphics-Driven Query Processing

Over the past decade, the database research community has directed considerable attention towards harnessing the power of GPUs in query processing engines. The proposed techniques have primarily focused on devising customized low-level mechanisms that utilize the raw hardware parallelism provided abundantly by GPU compute kernels. In this paper, we advocate a radically different approach - instead of dealing directly with hardware idiosyncrasies, to leverage the well-established graphics pipeline architecture baked into the GPU hardware. A variety of advantages accrue from this high-level abstraction: (a) Extracting the power of GPUs is outsourced to highly-optimized graphics drivers, thereby providing hardware-consciousness for free; (b) Query processing becomes agnostic to changes in GPU architectures (e.g. integrated vs discrete) and vendors, requiring only a change of drivers; (c) Contemporary graphics APIs also support a compute element, facilitating query operator designs that seamlessly straddle the compute and graphics worlds. As a proof of concept of the above vision, we implement here the workhorse Join and GroupBy operators using core graphics primitives. These implementations, based on the Vulkan API, have been evaluated over large benchmark databases on vanilla hybrid computing platforms. The experimental results indicate both substantive performance benefits (typically, around 2X faster) over existing approaches, as well as auto-tuned portability to new hardware platforms.

Vulkan all the way: Transitioning to a modern low-level graphics API in academia

For over two decades, the OpenGL API provided users with the means for implementing versatile, feature-rich, and portable real-time graphics applications. Consequently, it has been widely adopted by practitioners and educators alike and is deeply ingrained in many curricula that teach real-time graphics for higher education. Over the years, the architecture of graphics processing units (GPUs) incrementally diverged from OpenGL’s conceptual design. The more recently introduced Vulkan API provides a more modern, fine-grained approach for interfacing with the GPU, which allows a high level of controllability and, thereby, deep insights into the inner workings of modern GPUs. This property makes the Vulkan API especially well suitable for teaching graphics programming in university education, where fundamental knowledge shall be conveyed. Hence, it stands to reason that educators who have their students’ best interests at heart should provide them with corresponding lecture material. However, Vulkan is notoriously verbose and rather challenging for first-time users, thus transitioning to this new API bears a considerable risk of failing to achieve expected teaching goals. In this paper, we document our experiences after teaching Vulkan in both introductory and advanced graphics courses side-by-side with conventional OpenGL. A collection of surveys enables us to draw conclusions about perceived workload, difficulty, and students’ acceptance of either approach. In doing so, we identify suitable conditions and recommendations for teaching Vulkan to both undergraduate and graduate students.

LuisaRender

The advancements in hardware have drawn more attention than ever to high-quality offline rendering with modern stream processors, both in the industry and in research fields. However, the graphics APIs are fragmented and existing shading languages lack high-level constructs such as polymorphism, which adds complexity to developing and maintaining cross-platform high-performance renderers. We present LuisaRender 1 , a high-performance rendering framework for modern stream-architecture hardware. Our main contribution is an expressive C++-embedded DSL for kernel programming with JIT code generation and compilation. We also implement a unified runtime layer with resource wrappers and an optimized Monte Carlo renderer. Experiments on test scenes show that LuisaRender achieves much higher performance than existing research renderers on modern graphics hardware, e.g., 5--11× faster than PBRT-v4 and 4--16× faster than Mitsuba 3.

The use of modern computer technologies in the study of the Geometric Modeling course

The paper describes the educational discipline “Geometric modeling”, which is suitable for students of applied mathematics. The course covers spline representations and transfinite interpolation methods for modeling the shape of geometric objects, the methods for performing Boolean operations between objects and methods for visualizing the objects. Screenshots of some graphic windows of the developed visualization programs for supporting the implementation of individual assignments in the course are shown. The programs have web interfaces that use the WebGL graphics API in a browser, independent of an operating system, and are also available for smartphones and tablets. They provide graphical output for curves and surfaces in three-dimensional space, user interfaces for entering the initial data and interaction with geometric objects. Students only need to write their own calculation code for the programs in the JavaScript programming language. The interactive tools enable students to get practical skills in working with splines and visually analyze the impact of the input data on the final geometry. The individual assignments and programs are specially focused to stimulate students’ interests. The paper is addressed to teachers of the geometric modeling for mathematicians and computer scientists. It seems to be interesting to those who develop software interfaces for geometric modeling algorithms.

Integrating tsunami simulations in web applications using BROWNI, an open source client-side GPU-powered tsunami simulation library

Tsunami simulation software is a key component of state-of-the-art early warning systems but the inherent complexities in phases of installation, execution, pre and post-processing prevent their use in other areas of risk management such as communication and education. Recent advances in software and computational capacities such as the efficiency of GPU computing and the ubiquity of web browsers bring new opportunities to bridge the gap between expert and non-expert users. Here we present a Javascript library to enable a web browser to facilitate gathering and analyzing data from tsunami simulations, by means of interactive and efficient visualizations. At its core, the library uses WebGL, the browser’s standard 3D graphics API, to run GPU accelerated computations of a tsunami model. A far-field tsunami model is implemented (linear shallow water equations discretized on spherical coordinates), and its implementation is validated against real tsunami observations, and benchmarked with two other tsunami software-packages. Two software platforms that use this library are presented to illustrate the powerful applications that can be developed for risk communication and education. These applications are characterized by their interactivity and fast computations, which enable users to focus on the understanding of the phenomena of tsunami propagation and iterate quickly to assess different scenarios and potential implications to tsunami risk management. Some limitations on this approach are discussed, in aspects such as scalability, performance, multi-threading and batch-processing, that can be relevant for other users. In our experience, the before mentioned benefits very well compensate the discussed limitations for this kind of applications. The library has an open source license, and is meant to be imported without modifying its source code to facilitate the creation of new applications as the ones herein presented.

Cooperative Profile Guided Optimizations

AbstractExisting feedback‐driven optimization frameworks are not suitable for video games, which tend to push the limits of performance of gaming platforms and have real‐time constraints that preclude all but the simplest execution profiling. While Profile Guided Optimization (PGO) is a well‐established optimization approach, existing PGO techniques are ill‐suited for games for a number of reasons, particularly because heavyweight profiling makes interactive applications unresponsive. Adaptive optimization frameworks continually collect metrics that guide code specialization optimizations during program execution but have similarly high overheads. We emulate a system, which we call Cooperative PGO, in which the gaming platform collects piecemeal profiles by sampling in both time and space during actual gameplay across many users; stitches the piecemeal profiles together statistically; and creates policies to guide future gameplay. We introduce a three‐level hierarchical profiler that is well‐suited to graphics APIs, that commonly operates with no overhead and occasionally introduces an average overhead of less than 0.5% during periods of active profiling. This paper examines the practicality of Cooperative PGO using three PGOs as case studies. A PGO that exploits likely zeros is particularly effective, achieving an average speedup of 5%, with a maximum speedup of 15%, over a highly‐tuned baseline.

Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering

Mobile Augmented Reality (MAR) is designed to keep pace with high-end mobile computing and their powerful sensors. This evolution excludes users with low-end devices and network constraints. This article presents ModAR, a hybrid Android prototype that expands the MAR experience to the aforementioned target group. It combines feature-based image matching and pose estimation with fast rendering of 3D textured models. Planar objects of the real environment are used as pattern images for overlaying users’ meshes or the app’s default ones. Since ModAR is based on the OpenCV C++ library at Android NDK and OpenGL ES 2.0 graphics API, there are no dependencies on additional software, operating system version or model-specific hardware. The developed 3D graphics engine implements optimized vertex-data rendering with a combination of data grouping, synchronization, sub-texture compression and instancing for limited CPU/GPU resources and a single-threaded approach. It achieves up to 3× speed-up compared to standard index rendering, and AR overlay of a 50 K vertices 3D model in less than 30 s. Several deployment scenarios on pose estimation demonstrate that the oriented FAST detector with an upper threshold of features per frame combined with the ORB descriptor yield best results in terms of robustness and efficiency, achieving a 90% reduction of image matching time compared to the time required by the AGAST detector and the BRISK descriptor, corresponding to pattern recognition accuracy of above 90% for a wide range of scale changes, regardless of any in-plane rotations and partial occlusions of the pattern.

Assembly of Biomolecular Gigastructures and Visualization with the Vulkan Graphics API.

Building and displaying all-atom models of biomolecular structures with millions or billions of atoms, like virus particles or cells, remain a challenge due to the sheer size of the data, the required levels of automated building, and the visualization limits of today’s graphics hardware. Based on concepts introduced with the CellPack program, we report new algorithms to create such large-scale models using an intermediate coarse-grained “pet representation” of biomolecules with 1/10th the normal size. Pet atoms are placed such that they optimally trace the surface of the original molecule with just ∼1/50th the original atom number and are joined with covalent bonds. Molecular dynamics simulations of pet molecules allow for efficient packing optimization, as well as the generation of realistic DNA/RNA conformations. This pet world can be expanded back to the all-atom representation to be explored and visualized with full details. Essential for the efficient interactive visualization of gigastructures is the use of multiple levels of detail (LODs), where distant molecules are drawn with a heavily reduced polygon count. We present a grid-based algorithm to create such LODs for all common molecular graphics styles (including ball-and-sticks, ribbons, and cartoons) that do not require monochrome molecules to hide LOD transitions. As a practical application, we built all-atom models of SARS-CoV-2, HIV, and an entire presynaptic bouton with 1 μm diameter and 3.6 billion atoms, using modular building blocks to significantly reduce GPU memory requirements through instancing. We employ the Vulkan graphics API to maximize performance on consumer grade hardware and describe how to use the mmCIF format to efficiently store such giant models. An implementation is available as part of the YASARA molecular modeling and simulation program from www.YASARA.org. The free YASARA View program can be used to explore the presented models, which can be downloaded from www.YASARA.org/petworld, a Creative Commons platform for sharing giant biomolecular structures.

Α PATTERN-BASED AUGMENTED REALITY APPLICATION FOR THE DISSEMINATION OF CULTURAL HERITAGE

Abstract. Augmented Reality (AR) is more than an added value for Cultural Heritage (CH); it is vital for its sustainability, promotion and dissemination, increasing accessibility in CH even during difficult periods of time, like the Covid-19 pandemic. In order to be meaningful and engaging, an AR application should have the following characteristics: easiness of use, high-quality representations and compatibility. This paper presents a marker-less mobile AR application for the display and inspection of high-resolution 3D cultural assets, overlayed on a particular location in the real-world scene. Instead of predefined markers, an image captured by the user is exploited as a pattern for real-time feature matching, pose estimation and scene augmentation. Our approach is based on pure computer vision and photogrammetric techniques, implemented using native C++ and Java code for Android mobile platforms. It is built with the use of the OpenCV library and the OpenGL ES graphics API without any dependencies of AR Software Development Kits (SDKs). Therefore, it supports cross-vendor portability regarding mobile model devices and hardware specifications. The evaluation of the developed application examines the performance of various matching techniques and the overall responsiveness of processing and 3D rendering on mid-range and low-end smartphones. The results showcase the reliability and responsiveness of the pattern recognition as well as the potential of the 3D graphics engine to render and overlay complex 3D models balancing between visual quality and time. The proposed methodology is applied to the Ciborium of the church of St. Charalabos, located at St. Stephen’s Monastery in Meteora, Greece.

Guided Visibility Sampling++

Visibility computation is a common problem in the field of computer graphics. Examples include occlusion culling, where parts of the scene are culled away, or global illumination simulations, which are based on the mutual visibility of pairs of points to calculate lighting. In this paper, an aggressive from-region visibility technique called Guided Visibility Sampling++ (GVS++) is presented. The proposed technique improves the Guided Visibility Sampling algorithm through improved sampling strategies, thus achieving low error rates on various scenes, and being over four orders of magnitude faster than the original CPU-based Guided Visibility Sampling implementation. We present sampling strategies that adaptively compute sample locations and use ray casting to determine a set of triangles visible from a flat or volumetric rectangular region in space. This set is called a potentially visible set (PVS). Based on initial random sampling, subsequent exploration phases progressively grow an intermediate solution. A termination criterion is used to terminate the PVS search. A modern implementation using the Vulkan graphics API and RTX ray tracing is discussed. Furthermore, we show optimizations that allow for an implementation that is over 20 times faster than a naive implementation.

Некоторые аспекты визуализации трехмерных сферического и эллиптического пространств

In this paper we examine methods and algorithms for visualizing three-dimensional non-Euclidean spherical and elliptical spaces with a first-person view, from insider’s perspective: we propose an algorithm for implementing camera controls in such spaces, address an approach to visualizing properties of such spaces based on matrix and vector operations which enables us to perform some of the computations necessary for the visualization on GPUs using graphics APIs, describe an implementation of fog effects in such spaces for the purpose of simplifying navigation in them. An implementation of these approaches and algorithms that is intended for visualizing dynamic scenes in such spaces is presented.

MVC-3DC: Software architecture model for designing collaborative augmented reality and virtual reality systems

In this paper, software architecture model “MVC-3DC” for Collaborative Augmented and Virtual Reality Systems design is proposed. This model is the results of merging several aspects: Human-Computer Interaction (HCI), distribution systems, computer-supported cooperative work (CSCW) and new technologies such as augmented reality and virtual reality. MVC-3DC integrates collaboration principles between remote users. MVC-3DC allows a low dependency between components such as the core functions, 3D graphics API and data distribution modes. The proposed architectural model integrates simulation models, SDKs and algorithms for different nodes involved in a collaborative session. This facilitates interoperability and capability to manage heterogeneity and relationship between different nodes participating in the collaborative session. Finally, our model makes it possible to integrate other toolkits without completely changing the structure of collaboration model. A simple adaptation could be made.

The MG-RAST API explorer: an on-ramp for RESTful query composition

BackgroundThe MG-RAST API provides search capabilities and delivers organism and function data as well as raw or annotated sequence data via the web interface and its RESTful API. For casual users, however, RESTful APIs are hard to learn and work with.ResultsWe created the graphical MG-RAST API explorer to help researchers more easily build and export API queries; understand the data abstractions and indices available in MG-RAST; and use the results presented in-browser for exploration, development, and debugging.ConclusionsThe API explorer lowers the barrier to entry for occasional or first-time MG-RAST API users.

General-Purpose Computation on GPUs in the Browser Using gpu.js

gpu.js is a client-side general-purpose computing on graphics processing units (GPGPU) library for the browser written entirely in JavaScript. Unlike some of the earlier implementations of client-side GPGPU, gpu.js does not require browser support through an explicit GPGPU API or an installation of a custom native runtime browser extension to enable such support. This allows the library to run on all modern platforms, including on mobile devices such as smartphones. It achieves this by using the already widely adopted Web Graphics Library graphics API in a manner that it is not designed for by making use of JavaScript-to-Open GL Shading Language transpilation. The library abstracts away all the unnecessary implementation details of awkwardly performing GPGPU on a graphics API and, at the same time, provides an API that was designed specifically for GPGPU.

Shader components

Modern game engines seek to balance the conflicting goals of high rendering performance and productive software development. To improve CPU performance, the most recent generation of real-time graphics APIs provide new primitives for performing efficient batch updates to shader parameters. However, modern game engines featuring large shader codebases have struggled to take advantage of these benefits. The problem is that even though shader parameters can be organized into efficient modules bound to the pipeline at various frequencies, modern shading languages lack corresponding primitives to organize shader logic (requiring these parameters) into modules as well. The result is that complex shaders are typically compiled to use a monolithic block of parameters, defeating the design, and performance benefits, of the new parameter binding API. In this paper we propose to resolve this mismatch by introducing shader components , a first-class unit of modularity in a shader program that encapsulates a unit of shader logic and the parameters that must be bound when that logic is in use. We show that by building sophisticated shaders out of components, we can retain essential aspects of performance (static specialization of the shader logic in use and efficient update of parameters at component granularity) while maintaining the modular shader code structure that is desirable in today's high-end game engines.

Visualising higher-dimensional space-time and space-scale objects as projections to ℝ3

Objects of more than three dimensions can be used to model geographic phenomena that occur in space, time and scale. For instance, a single 4D object can be used to represent the changes in a 3D object’s shape across time or all its optimal representations at various levels of detail. In this paper, we look at how such higher-dimensional space-time and space-scale objects can be visualised as projections from ℝ4to ℝ3. We present three projections that we believe are particularly intuitive for this purpose: (i) a simple ‘long axis’ projection that puts 3D objects side by side; (ii) the well-known orthographic and perspective projections; and (iii) a projection to a 3-sphere (S3) followed by a stereographic projection to ℝ3, which results in an inwards-outwards fourth axis. Our focus is in using these projections from ℝ4to ℝ3, but they are formulated from ℝnto ℝn−1so as to be easily extensible and to incorporate other non-spatial characteristics. We present a prototype interactive visualiser that applies these projections from 4D to 3D in real-time using the programmable pipeline and compute shaders of the Metal graphics API.

Interactive GPU-based Visualization of Large Dynamic Particle Data

Prevalent types of data in scientific visualization are volumetric data, vector field data, and particle-based data. Particle data typically originates from measurements and simulations in various fields, such as life sciences or physics. The particles are often visualized directly, that is, by simple representants like spheres. Interactive rendering facilitates the exploration and visual analysis of the data. With increasing data set sizes in terms of particle numbers, interactive high-quality visualization is a challenging task. This is especially true for dynamic data or abstract representations that are based on the raw particle data. This book covers direct particle visualization using simple glyphs as well as abstractions that are application-driven such as clustering and aggregation. It targets visualization researchers and developers who are interested in visualization techniques for large, dynamic particle-based data. Its explanations focus on GPU-accelerated algorithms for high-performance rendering and data processing that run in real-time on modern desktop hardware. Consequently, the implementation of said algorithms and the required data structures to make use of the capabilities of modern graphics APIs are discussed in detail. Furthermore, it covers GPU-accelerated methods for the generation of application-dependent abstract representations. This includes various representations commonly used in application areas such as structural biology, systems biology, thermodynamics, and astrophysics.

VGASA: Adaptive Scheduling Algorithm of Virtualized GPU Resource in Cloud Gaming

As the virtualization technology for GPUs matures, cloud gaming has become an emerging application among cloud services. In addition to the poor default mechanisms of GPU resource sharing, the performance of cloud games is inevitably undermined by various runtime uncertainties such as rendering complex game scenarios. The question of how to handle the runtime uncertainties for GPU resource sharing remains unanswered. To address this challenge, we propose vGASA, a virtualized GPU resource adaptive scheduling algorithm in cloud gaming. vGASA interposes scheduling algorithms in the graphics API of the operating system, and hence the host graphic driver or the guest operating system remains unmodified. To fulfill the service level agreement as well as maximize GPU usage, we propose three adaptive scheduling algorithms featuring feedback control that mitigates the impact of the runtime uncertainties on the system performance. The experimental results demonstrate that vGASA is able to maintain frames per second of various workloads at the desired level with the performance overhead limited to 5-12 percent.