Real-time Ray Tracing Research Articles

Deep learning-based real-time ray tracing technology in games

Abstract. In recent years, deep learning-based techniques have revolutionized real-time ray tracing for gaming, significantly enhancing visual fidelity and rendering performance. This paper reviews various state-of-the-art methods, including the use of Generative Adversarial Networks (GANs) for realistic shading, the use of neural temporal adaptive sampling, the use of subpixel sampling reconstruction, and the use of neural scene representation. Key findings highlight improvements in perceived realism, temporal stability, image fidelity, and computational efficiency. Techniques such as neural intersection functions and spatiotemporal reservoir resampling further optimize rendering speed and memory usage. Additionally, approaches like adaptive sampling and neural denoising using layer embeddings contribute to reduced noise and enhanced image clarity. Collectively, these advancements make real-time ray tracing more feasible for high-fidelity gaming applications, offering enhanced graphics without compromising performance. These improvements are significant. My analysis underscores the critical role of deep learning in overcoming traditional ray tracing challenges, paving the way for more immersive and responsive gaming experiences. Furthermore, these innovations suggest a promising future for integrating advanced ray tracing techniques into a broader range of interactive media, ensuring both visual excellence and operational efficiency.

HLS-Based Approach for Embedded Real-Time Ray Tracing in Wireless Communications

HLS-Based Approach for Embedded Real-Time Ray Tracing in Wireless Communications

Deep learning-driven optimization of real-time ray tracing for enhanced immersive virtual reality

Photorealistic rendering is essential for immersive virtual reality and real-time ray tracing is required for this, yet the necessary computational load has so far hindered its use in scenarios where a fast response and high frame rates are paramount. To overcome this challenge, we present a deep learning-based model that optimises the computational load of ray tracing through utilising convolutional neural networks (CNNs) to predict lightscene interactions. We achieve this by training CNNs from datasets with offline ray-traced images. Using this learning process, we approximate the contents of light transport simulations from scene point sampling. The use of this optimised algorithm in virtual reality scenes, thus, enables renderings of more complex scenes, with dynamic lighting, complex geometry and interactive elements under high frame rates, keeping users immersed and responsive to events. Our results show that this method achieves visual similarities to traditional ray tracing, which is photorealistic rendering, but can be performed in real-time in virtual reality. We anticipate this work leads to many potential applications of ray tracing in many scenarios in virtual reality, such as gaming, architectural rendering and training.

Using Real-Time Ray Tracing in Game Action-Adventure ANGKARA "The Rise of Asura"

The pressing problems facing the games industry in Indonesia are first, the lack of developers who can meet the needs of the local market and the lack of adaptation of local content in games. Second, how to improve the optimization of the Ray Tracing process? To overcome these challenges, this research aims to design and build an action-adventure game model using Unreal Engine 5. By developing the Real-Time Ray Tracing (RTRT) feature. The game design will also utilize the latest technology such as Ray Tracing, and artificial intelligence to create an immersive and realistic gaming experience. The method used in this research is GDLC (Game Development Life Cycle). The results of research using black box testing with components, character responsiveness, basic attacks, interaction with the environment, defense, enemy behavior, and optimization of Ray Tracing state that it has been validated.

Single-layer waveguide with compound metasurfaces for highly efficient and chromatic-aberration-free augmented reality near-eye displays

Augmented reality (AR) near-eye displays seamlessly integrate virtual information with real world, in which the optical waveguide is the key component functioned as a coupler. Due to strong chromatic aberration, independent waveguides are always indispensable to separate the three primary colors that inevitably brings larger and thicker appearance. To address this, this paper proposes a new chromatic-aberration-free single-layer waveguide for AR displays by integrating a double-layer grating metasurface (MS) and the covered cylindrical MSs. The double-layer grating MS breaks through the limitations on a narrow wavelength band, enabling high diffraction efficiency for incident light deflection across the entire visible spectrum. Further, the cylindrical MSs provide accurate control over incident angles, not only overcoming the challenge of total reflection but also effectively eliminating chromatic aberration. Design results show that the proposed structure achieves an impressive diffraction efficiency of over 80 % for blue and green light and over 90 % for red light with an outstanding diffraction angular difference lower than 2.5 deg. Real-time ray tracing simulation demonstrates the capability of the designed AR engine to achieve high-resolution full-color imaging. The compound MSs enable precise light and color control for a single-layer waveguide coupler with exceptional diffraction efficiency and minimal chromatic aberration, which show great potential for cutting-edge AR applications.

Ray tracing-based construction of 3D background model for real-time stereoscopic rendering of live immersive video

Immersive video stored in multiview video-plus-depth format can provide viewers with vivid immersive experiences. However, rendering such video in real time in immersive environments remains a challenging task due to the high resolution and refresh rate demanded by recent extended reality displays. An essential issue in this immersive rendering is the disocclusion problem that inevitably occurs when virtual views are synthesized via the de facto standard 3D warping technique. In this paper, we present a novel virtual view synthesis framework that, from a live immersive video stream, renders stereoscopic images in real time for a freely moving virtual viewer. The main difference from previous approaches is that the surrounding background environment of the immersive video’s virtual scene is progressively reproduced on the fly directly in the 3D space while the input stream is being rendered. To allow this, we propose a new 3D background modeling scheme that, based on GPU-accelerated real-time ray tracing, efficiently and incrementally builds the background model in compact 3D triangular mesh. Then, we demonstrate that the 3D background environment can effectively alleviate the critical disocclusion problem in the immersive rendering, eventually reducing spatial and temporal aliasing artifacts. It is also suggested that the 3D representation of background environment enables extension of the virtual environment of immersive video by interactively adding 3D visual effects during rendering.

Specular Path Generation and Near-Reflective Diffraction in Interactive Acoustical Simulations.

Most systems for simulating sound propagation in a virtual environment for interactive applications use ray- or path-based models of sound. With these models, the "early" (low-order) specular reflection paths play a key role in defining the "sound" of the environment. However, the wave nature of sound, and the fact that smooth objects are approximated by triangle meshes, pose challenges for creating realistic approximations of the reflection results. Existing methods which produce accurate results are too slow to be used in most interactive applications with dynamic scenes. This paper presents a method for reflections modeling called spatially sampled near-reflective diffraction (SSNRD), based on an existing approximate diffraction model, Volumetric Diffraction and Transmission (VDaT). The SSNRD model addresses the challenges mentioned above, produces results accurate to within 1-2 dB on average compared to edge diffraction, and is fast enough to generate thousands of paths in a few milliseconds in large scenes. This method encompasses scene geometry processing, path trajectory generation, spatial sampling for diffraction modeling, and a small deep neural network (DNN) to produce the final response of each path. All steps of the method are GPU-accelerated, and NVIDIA RTX real-time ray tracing hardware is used for spatial computing tasks beyond just traditional ray tracing.

Neural Temporal Denoising for Indirect Illumination.

Various temporal denoising methods have been proposed to clean up the noise for real-time ray tracing (RTRT). These methods rely on the temporal correspondences of pixels between the current and previous frames, i.e. per-pixel screen-space motion vectors. However, the state-of-the-art temporal reuse methods with traditional motion vectors cause artifacts in motion occlusions. We accordingly propose a novel neural temporal denoising method for indirect illumination of Monte Carlo (MC) ray tracing at 1 sample per pixel. Based on end-to-end multi-scale kernel-based reconstruction, we apply temporally reliable dual motion vectors to facilitate better reconstruction of the occlusions, and also introduce additional motion occlusion loss to reduce ghosting artifacts. Experiments show that our method significantly reduces the over-blurring and ghosting artifacts while generating high-quality images at real-time rates.

Real-time Ray Tracing for Cardiothoracic Imaging.

Real-time Ray Tracing for Cardiothoracic Imaging.

PVLI: potentially visible layered image for real-time ray tracing

PVLI: potentially visible layered image for real-time ray tracing

Analysis of Ray Tracing Methodology and Techniques and its Distinction from other Render Models

Abstract: In 3D computer-generated graphics, ray tracing is a form of rendering technique for calculating light transport for the purpose of giving a photo-realistic image. This survey paper mainly focuses on the brief working of ray tracing methodology and various techniques available to achieve this. This helps to understand one how ray tracing can be applied in the field of computer graphics to produce stunning realistic renders. At present time ray tracing is primarily available for gaming (real-time ray tracing) and also for visual effects in cinema industry. This paper also gives a detailed study about hardware and software components used for ray tracing

Application of real-time rendering technology to archaeological heritage virtual reconstruction: the example of Casas del Turuñuelo (Guareña, Badajoz, Spain)

Highlights: The use of real-time rendering with ray tracing technology as a tool for heritage virtual reconstruction is proposed. The possibilities that the use of next-generation video game engines, specifically Unreal Engine, offer are evaluated in terms of their application in heritage virtualisation. The first results of the virtual reconstruction of the Tartessian site of Casas del Turuñuelo are presented, after using real-time ray tracing technology as a research method to create and review architectural hypotheses. Abstract: Virtual reconstruction has become a fundamental tool to study and analyse archaeological heritage, given its usefulness for both research and dissemination. Although the discipline has advanced exponentially in recent years, the workflow used in most jobs is still based on the offline methodology as the preferred rendering engine. In contrast, this paper proposes the substitution of this methodology with the new ray tracing in real-time rendering technology; specifically, the authors used Unreal Engine to develop virtual reconstruction work as a research tool during the excavation of an archaeological site, as well as to disseminate the results of the study of each phase. The aim is to exploit the advantages of the immediacy of calculating high-quality and realistic lighting and materials, as well as the interaction and immersion in the virtual model that this system for the development of video games offers. This paper highlights: a) the benefits detected when using real-time technology in heritage reconstruction during the work carried out to date, and b) its limitations and its future evolution with the development of the technology. To demonstrate the usefulness of this tool, the authors present the reconstruction project of the Casas del Turuñuelo site (Guareña, Badajoz). It is one of the best preserved protohistoric sites in the Western Mediterranean, which is why applying this technology to this case study was considered appropriate. The excellent architectural preservation of the Casas del Turuñuelo building is an extraordinary example to assess the usefulness of applying video game engines to heritage reconstruction. This settlement is one of the first known examples of this technology being applied to heritage, specifically, to the virtualisation of an archaeological site under excavation. This methodology and its improvements will be applied to the virtual reconstruction of this project as the excavation of this site advances; thus, one of the main outreach tools developed within the framework of Building Tartessos project will be made available to users as a final product.

Technology-driven Virtual Production

The “subversive” technological drive of game engines for virtual film production is gradually being valued by filmmakers. This technical advantage is manifested in the game engine’s (1) real-time rendering of final pixels (speed and image quality), (2) “post-production in advance” and multi-pipeline for project production, (3) open-source code sharing and plug-ins for VFX. Based on the advantages, this article takes the mainstream Unreal Engine (UE) and Unity as the research objects to clarify the application prospects of game engines in four aspects: digital humans with high fidelity, real-time ray tracing in complex scenes, in-camera VFX, and remote collaboration in the post-pandemic era.

Sensor-aware frontier exploration and mapping with application to thermal mapping of building interiors

The combination of simultaneous localization and mapping (SLAM) and frontier exploration enables a robot to traverse and map an unknown area autonomously. Most prior autonomous SLAM solutions utilize information only from depth sensing devices. However, in situations where the main goal is to collect data from auxiliary sensors such as thermal camera, existing approaches require two passes: one pass to create a map of the environment and another to collect the auxiliary data, which is time consuming and energy inefficient. We propose a sensor-aware frontier exploration algorithm that enables the robot to perform map construction and auxiliary data collection in one pass. Specifically, our method uses a real-time ray tracing technique to construct a map that encodes unvisited locations from the perspective of auxiliary sensors rather than depth sensors; this encourages the robot to fully explore those areas to complete the data collection task and map making in one pass. Our proposed exploration framework is deployed on a LoCoBot with the task to collect thermal images from building envelopes. We validate with experiments in both multi-room commercial buildings and cluttered residential buildings. Using a metric that evaluates the coverage of sensor data, our method significantly outperforms the baseline method with a naive SLAM algorithm.

상용 3 차원 고속연산 엔진과 지도정보 연동 도심 환경의 GNSS 다중경로신호 제거 및 실시간 항법해 개선 연구

This paper presents a technique to improve the accuracy of navigation of a satellite navigation system in the urban environment by effectively detecting and removing multipath signals in real time using 3D map information. To improve the satellite navigation accuracy, we designed a system that can implement an effective real-time ray-tracing calculation with 3D high-speed computation engine. In this process, objective multipath satellites can be identified and separated by collecting the actual satellite data around a complex urban environment and around the partial shadow areas during the satellite navigation. The Unity3D Engine ensures a real-time and efficient computing performance in large sized complex map environments and multiple access situations. By adapting the functionality of Unity3D, the GPU computation for satellite visibility is efficiently implemented in a complex map environment, resulting into the implementation of a real-time ray tracing function, even in a mobile platform. Consequently, the computation time for detecting a multipath signal with one set of satellite data is shortened. In addition, the root mean square error in estimating the horizontal position using the GPS L1 pseudorange least square navigation solution, which eliminates multipath signals, is reduced.

Real-time, model-aided ranging for small scale operations in the Beaufort Sea, an ice-covered, double ducted environment

This talk will cover operational methods and results for vehicle navigation from the Ice Exercise in March 2020 (ICEX20), in the Beaufort Sea. For short range acoustic transmissions, the total ice cover and the double ducted environment co-create complex multipath uncertainty. We assumed that the horizontal group velocity between source and receiver was smoothly varying, and embedded a “Virtual Ocean” with a real-time ray tracing engine to predict the horizontal group velocity for range estimation. This model-aided psuedorange approach, which enabled a successful vehicle recovery, favored the least multipath possible such that psuedoranges tended to overestimate the GPS-derived range by roughly 10 m. In post-processing, we modify the group velocity estimation method to consider varying degrees of multipath and achieve a mean absolute error of roughly 4 plus or minus 4 meters, rivaling GPS performance. To our knowledge, these results are the first field experiment to demonstrate a real-time, model-based data processing to constrain ranging error for underwater navigation. [Work supported by ONR & the NDSEG fellowship.]

A hardware accelerated unstructured overset method to simulate turbulent fluid flow

Hardware acceleration consists of offloading computational work to devices such as graphics processing units (GPUs) to produce overall speed-up. Algorithms and numerical methods must be constructed to suit the available hardware in order to effectively produce speed-up. In this work a numerical method is presented which can effectively use hardware acceleration to simulate incompressible turbulent fluid flow. The method is an unstructured overset method where unstructured meshes are attached to individual bodies and connected throughout the flow domain to produce a single domain solution through an overset assembly process. The unstructured overset method shown in Horne and Mahesh [1] and Horne and Mahesh [2] was found capable of scaling to O(105) computational cores for O(105) moving bodies in turbulent flow fields while producing accurate flow results. This highly scalable method is modified and extended to effectively utilize on-node hardware acceleration. Overset assembly algorithms which use hardware acceleration are presented based on successful accelerated algorithms in real-time ray tracing and computational geometry. Timing results for core overset assembly operations are presented showing a maximum O(100x) speedup when using hardware acceleration. A novel method for turbulent fluid flow is presented which utilizes over-decomposition of the flow domain to produce task-parallelism allowing asynchronous calculation of the different steps of the method while also providing overlap between data transfer and computation. A mixed precision solver is utilized which provides a balance between optimal performance and numerical accuracy. A cost effective and accurate artificial compressibility pressure regularization is used which has minimal memory complexity and minimizes computational cost while maintaining accuracy. A primal-dual Laplacian operator is introduced which produces accurate results on skewed meshes. Results for canonical flow cases with overset meshes are shown illustrating the method's accuracy and numerical properties. Substantial speed-up is demonstrated for the numerical method reaching upwards of 50 times as fast as the non-accelerated method for high cell loadings.

BRDF importance sampling for polygonal lights

With the advent of real-time ray tracing, there is an increasing interest in GPU-friendly importance sampling techniques. We present such methods to sample convex polygonal lights approximately proportional to diffuse and specular BRDFs times the cosine term. For diffuse surfaces, we sample the polygons proportional to projected solid angle. Our algorithm partitions the polygon suitably and employs inverse function sampling for each part. Inversion of the distribution function is challenging. Using algebraic geometry, we develop a special iterative procedure and an initialization scheme. Together, they achieve high accuracy in all possible situations with only two iterations. Our implementation is numerically stable and fast. For specular BRDFs, this method enables us to sample the polygon proportional to a linearly transformed cosine. We combine these diffuse and specular sampling strategies through novel variants of optimal multiple importance sampling. Our techniques render direct lighting from Lambertian polygonal lights with almost no variance outside of penumbrae and support shadows and textured emission. Additionally, we propose an algorithm for solid angle sampling of polygons. It is faster and more stable than existing methods.

Rendering transparent objects with caustics using real-time ray tracing

After the release of the NVIDIA RTX platform, ray tracing methods are available in real-time rendering and improves the render quality. Therefore, realistic results can be produced in real time by combining ray tracing with rasterization techniques. However, rendering transparent objects with roughness and caustics in real time is still a challenge. We focus on these points and propose a model to render transparent objects based on real-time ray tracing. The model consists of two parts: Iterative Ray Tracing Transparency (IRTT) and Light Half Path Caustics (LHPC). IRTT approximates accurate reflections and refractions by tracing rays, computes volumetric absorption using the length that a ray propagates inside a transparent object, and simulates the rough transparency by bending the normal vector. Meanwhile, LHPC approximates caustics with a method named Local Light Tracing (LLT), where the main task is finding the point of incidence by tracing rays in the direction of the light source. After, a denoiser assisted by a distribution method is applied to suppress the noise. In the experiments, we design a combined pipeline and compare it with our model in terms of performance and render quality. In addition, our results are also compared with rasterization techniques. The experimental results reveal that our model improves the quality of the rendered image and offers a playable framerate.

An Implementation of Multi-Chip Architecture for Real-Time Ray Tracing Based on Parallel Frame Rendering

In this paper, we propose a multi-chip architecture based on parallel frame rendering suitable for real-time ray tracing in dynamic scenes. In multi-chip architecture with the commonly used screen partitioning method, the acceleration structure data, such as a tree, updated in a dynamic scene must be transmitted to each chip. In the proposed frame division method, a tree build and ray tracing are performed on the same chip, and each frame is allocated to a predesignated multi-chip. Thus, the proposed method can achieve scalable performance improvement not only of ray tracing but also of a tree build. We implemented a multi-chip architecture on three field-programmable gate array (FPGA) boards and built 12 ray-tracing cores in the FPGA chip of each board. This configuration means that the inter-chip operates using the frame division method, while the inner chip operates using the screen partitioning method. The results of experiments showed that the proposed multi-chip architecture improved frames per second (FPS) performance by an average of 2.83 times compared to a single-chip architecture.