Global Illumination Research Articles

Technical note: Interest of focused fields in post-mortem computed tomography using photorealistic images for age at death estimation from the pubic symphysis

IntroductionThe morphological assessment of the pubic symphysis using the Suchey-Brooks method is considered a reliable age at death indicator. Age at death estimation methods can be adapted to the images obtained from post-mortem computed tomography (PMCT). The aim of this study is to evaluate the utility of pubic symphysis photorealistic images obtained through Global illumination rendering (GIR) for age at death estimation from whole-body PMCT and from focused PMCT on the pubic bone. Materials and MethodsWe performed virtual age at death estimation using the Suchey Brooks method from both the whole-body field of view (Large Field of View: LFOV) and the pubis-focused field of view (Small and Field of View: SFOV) of 100 PMCT. The 3D photorealistic images were evaluated by three forensic anthropologists and the results were statistically evaluated for accuracy of the two applied PMCT methods and the intra- and inter-observer errors. ResultsWhen comparing the two acquisitions of PMCT, the accuracy rate reaches 98.5% when using a pubic-focused window (SFOV) compared to 86% with a whole-body window (LFOV). Additionally, the intra- and inter-observer variability has demonstrated that the focused window provides better repeatability and reproducibility. ConclusionAdding a pubic-focused field of view to standard PMCT and processing it with GIR appears to be an applicable technique that increases the accuracy rate for age at death estimation from the pubic symphysis.

Neural Global Illumination: Interactive Indirect Illumination Prediction under Dynamic Area Lights.

We propose neural global illumination, a novel method for fast rendering full global illumination in static scenes with dynamic viewpoint and area lighting. The key idea of our method is to utilize a deep rendering network to model the complex mapping from each shading point to global illumination. To efficiently learn the mapping, we propose a neural-network-friendly input representation including attributes of each shading point, viewpoint information, and a combinational lighting representation that enables high-quality fitting with a compact neural network. To synthesize high-frequency global illumination effects, we transform the low-dimension input to higher-dimension space by positional encoding and model the rendering network as a deep fully-connected network. Besides, we feed a screen-space neural buffer to our rendering network to share global information between objects in the screen-space to each shading point. We have demonstrated our neural global illumination method in rendering a wide variety of scenes exhibiting complex and all-frequency global illumination effects such as multiple-bounce glossy interreflection, color bleeding, and caustics.

Mathematical Modelling in Light and Engineering Education

The development of computer technology and mathematical methods for solving the global illumination equation gave rise to computer-aided lighting design systems, which in turn led to a genuine revolution in the practice of lighting design. Deep use of information technologies is possible only with a developed mathematical description of the light field in a lighting installation. It is the theory of the light field, based on the theory of radiation transfer, that allows us to present all lighting technology from a single conceptual point of view. The mathematical apparatus of the theory of the light field is the basis of the algorithms of all modern programs for the calculation of lighting installations, lighting devices, radiation, reflection, and transmission of light by real objects. As a result, there was a need for a significant change in light and engineering education. At the subdepartment of Light and Engineering of the NRU “MEI”, following modern trends, new and new courses have appeared and are appearing, allowing students to obtain relevant knowledge. One of the new courses planned at the subdepartment is a course of mathematical modelling in light and engineering. It involves an in-depth study of the internal structure of computer-aided design systems, which will allow obtaining specialists capable of solving non-standard tasks, creating funds for solving these tasks and conducting research.

The Impact of Target Object Size and Dimensionality on the Degree of Adaptation to Global Illumination

ABSTRACT Chromatic adaptation is an important visual phenomenon that specifically influences color appearance under various illumination conditions. There were extensive studies investigating the impact of the illumination condition and the surrounding condition on chromatic adaptation. However, past studies hardly consider the properties of the stimulus object such as color, size, dimensions, etc. In this study, the impact of the size and dimensionality of the stimulus (target object) on the degree of adaptation was investigated. A series of achromatic matching experiments were conducted in a lighting booth, with the adapting field varying in chromaticity and the object varying in size and dimensionality, to collect corresponding color data. Results indicate that the properties of the stimulus, such as the dimensionality and size, can significantly influence the adaptation state, independent of illumination chromaticity. But to apply the results in the real application may be restricted to the specific setup (used in this study) with a separately illuminated stimulus, which needs further investigation.

The role of different light settings on the perception of realism in virtual replicas in immersive Virtual Reality

Immersive Virtual Reality (IVR) provides a platform where the real world can be replicated to a point where users can act and react in the virtual world as they would in reality. However, rendering visual stimuli is computationally heavy. Thus, optimizations must be done to take advantage of computational systems by studying our perception of reality. This study investigated parameters related to light rendering (Global Illumination, Ambient Occlusion, Screen Space Reflections (SSR) and Direct Shadows) in real-time in a virtual replica of a real place using IVR. Participants experienced both virtual and real rooms with only one flashlight and changed the quality settings of the considered parameters so that their sense of reality would be the closest to the one they felt when they experienced the real room. Participants were given a budget to drive them to prioritize what parameters, and their level of quality, are the most important for their sense of reality. Results indicated that participants considered Global Illumination the most important factor, closely followed by Direct Shadows. Ambient Occlusion and Reflections (Screen Space Reflections) were the less prioritized parameters. We conclude that in a lighting setting where only dynamic lights are used, Global Illumination and Direct Shadows should be prioritized over SSR Reflections and Ambient Occlusion when computational power is limited.

Jrender: An efficient differentiable rendering library based on Jittor

Differentiable rendering has been proven as a powerful tool to bridge 2D images and 3D models. With the aid of differentiable rendering, tasks in computer vision and computer graphics could be solved more elegantly and accurately. To address challenges in the implementations of differentiable rendering methods, we present an efficient and modular differentiable rendering library named Jrender based on Jittor. Jrender supports surface rendering for 3D meshes and volume rendering for 3D volumes. Compared with previous differentiable renderers, Jrender exhibits a significant improvement in both performance and rendering quality. Due to the modular design, various rendering effects such as PBR materials shading, ambient occlusions, soft shadows, global illumination, and subsurface scattering could be easily supported in Jrender, which are not available in other differentiable rendering libraries. To validate our library, we integrate Jrender into applications such as 3D object reconstruction and NeRF, which show that our implementations could achieve the same quality with higher performance.

World‐Space Spatiotemporal Path Resampling for Path Tracing

AbstractWith the advent of hardware‐accelerated ray tracing, more and more real‐time rendering applications tend to render images with ray‐traced global illumination (GI). However, the low sample counts at real‐time framerates bring enormous challenges to existing path sampling methods. Recent work (ReSTIR GI) samples indirect illumination effectively with a dramatic bias reduction. However, as a screen‐space based path resampling approach, it can only reuse the path at the first bounce and brings subtle benefits for complex scenes. To this end, we propose a world‐space based spatiotemporal path resampling approach. Our approach caches more path samples into a world‐space grid, which allows reusing sub‐path starting from non‐primary path vertices. Furthermore, we introduce a practical normal‐aware hash grid construction approach, providing more efficient candidate samples for path resampling. Eventually, our method achieves improvements ranging from 16.6% to 41.9% in terms of mean squared errors (MSE) compared against the previous method with only 4.4% ∼ 8.4% extra time cost.

Neural Shading Fields for Efficient Facial Inverse Rendering

AbstractGiven a set of unstructured photographs of a subject under unknown lighting, 3D geometry reconstruction is relatively easy, but reflectance estimation remains a challenge. This is because it requires disentangling lighting from reflectance in the ambiguous observations. Solutions exist leveraging statistical, data‐driven priors to output plausible reflectance maps even in the under‐constrained single‐view, unknown lighting setting. We propose a very low‐cost inverse optimization method that does not rely on data‐driven priors, to obtain high‐quality diffuse and specular, albedo and normal maps in the setting of multi‐view unknown lighting. We introduce compact neural networks that learn the shading of a given scene by efficiently finding correlations in the appearance across the face. We jointly optimize the implicit global illumination of the scene in the networks with explicit diffuse and specular reflectance maps that can subsequently be used for physically‐based rendering. We analyze the veracity of results on ground truth data, and demonstrate that our reflectance maps maintain more detail and greater personal identity than state‐of‐the‐art deep learning and differentiable rendering methods.

IBL‐NeRF: Image‐Based Lighting Formulation of Neural Radiance Fields

AbstractWe propose IBL‐NeRF, which decomposes the neural radiance fields (NeRF) of large‐scale indoor scenes into intrinsic components. Recent approaches further decompose the baked radiance of the implicit volume into intrinsic components such that one can partially approximate the rendering equation. However, they are limited to representing isolated objects with a shared environment lighting, and suffer from computational burden to aggregate rays with Monte Carlo integration. In contrast, our prefiltered radiance field extends the original NeRF formulation to capture the spatial variation of lighting within the scene volume, in addition to surface properties. Specifically, the scenes of diverse materials are decomposed into intrinsic components for rendering, namely, albedo, roughness, surface normal, irradiance, and prefiltered radiance. All of the components are inferred as neural images from MLP, which can model large‐scale general scenes. Especially the prefiltered radiance effectively models the volumetric light field, and captures spatial variation beyond a single environment light. The prefiltering aggregates rays in a set of predefined neighborhood sizes such that we can replace the costly Monte Carlo integration of global illumination with a simple query from a neural image. By adopting NeRF, our approach inherits superior visual quality and multi‐view consistency for synthesized images as well as the intrinsic components. We demonstrate the performance on scenes with complex object layouts and light configurations, which could not be processed in any of the previous works.

Reordering and synchronization of electrical turbulence in cardiac tissue through global and partial optogenetical illumination.

Electrical turbulence in the heart is considered the culprit of cardiac disease, including the fatal ventricular fibrillation. Optogenetics is an emerging technology that has the capability to produce action potentials of cardiomyocytes to affect the electric wave propagation in cardiac tissue, thereby possessing the potential to control the turbulence, by shining a rotating spiral pattern onto the tissue. In this paper, we present a method to reorder and synchronize electrical turbulence through optogenetics. A generic two-variable reaction-diffusion model and a simplified three-variable ionic cardiac model are used. We discuss cases involving either global or partial illumination.

Low illumination image enhancement based on attention mechanism and global illumination estimation

Aiming at the problems of insufficient or excessive local brightness enhancement, color distortion, and excessive noise in the existing low-light image enhancement algorithms, a low-light image enhancement method combining attention mechanism and global illumination estimation is proposed. First, the illumination distribution map of low illumination image is obtained through the illumination distribution estimation network coupled with the attention gate mechanism. Then, the weight of the light distribution map is learned in the feature attention module. Finally, the image details are fused by the detail reconstruction module to create improved image. According to the experimental results, this method may effectively improve image brightness, contrast, and color in subjective visual effects while also enhancing objective evaluation indicators like PSNR, SSIM, and MSE when compared to some conventional methods.

NeLT: Object-Oriented Neural Light Transfer

This article presents object-oriented neural light transfer (NeLT), a novel neural representation of the dynamic light transportation between an object and the environment. Our method disentangles the global illumination of a scene into individual objects’ light transportation represented via neural networks, then composes them explicitly. It therefore enables flexible rendering with dynamic lighting, cameras, materials, and objects. Our rendering features various important global illumination effects, such as diffuse illumination, glossy illumination, dynamic shadowing, and indirect illumination, which completes the capability of existing neural object representation. Experiments show that NeLT does not require path tracing or shading results as input but achieves rendering quality comparable to state-of-the-art rendering frameworks, including the recent deep learning based denoisers.

Effect-based Multi-viewer Caching for Cloud-native Rendering

With cloud computing becoming ubiquitous, it appears as virtually everything can be offered as-a-service. However, real-time rendering in the cloud forms a notable exception, where the cloud adoption stops at running individual game instances in compute centers. In this paper, we explore whether a cloud-native rendering architecture is viable and scales to multi-client rendering scenarios. To this end, we propose world-space and on-surface caches to share rendering computations among viewers placed in the same virtual world. We discuss how caches can be utilized on an effect-basis and demonstrate that a large amount of computations can be saved as the number of viewers in a scene increases. Caches can easily be set up for various effects, including ambient occlusion, direct illumination, and diffuse global illumination. Our results underline that the image quality using cached rendering is on par with screen-space rendering and due to its simplicity and inherent coherence, cached rendering may even have advantages in single viewer setups. Analyzing the runtime and communication costs, we show that cached rendering is already viable in multi-GPU systems. Building on top of our research, cloud-native rendering may be just around the corner.

Design of large-angle distortion-free and spot adjustable LED spot array projector based on microlens array and eyepiece

The spot array projector has long been a challenging and intriguing research focus in the fields of projection and lighting applications. In the current literature, there is a lack of a comprehensive and detailed presentation on the design and analysis method for a spot array generator based on the structure of imaging eyepiece and MLA. We present a novel design and optimization method for a large-angle, distortion-free and spot adjustable LED spot array projector that is composed of an eyepiece, two microlens arrays (MLAs), and a micro-LED array (MLEDA). The eyepiece system is optimized using imaging optical methods to project sub-beams to the target plane with a large angle. The sub-lens of condenser MLA is also optimized using imaging optical methods to refocus the collimating beam and match the numerical aperture (NA) with the eyepiece, and the sub-lens of the collimating MLA is acquired by using simulated annealing (SA) global illumination optics optimization method to achieve collimation and far-field homogenization. The predistortion MLEDA and the MLAs are proposed and implemented by the radial basis function (RBF) interpolation method, which correct the large-angle distortion introduced by the eyepiece. Both near-field and far-field applications can be realized by the proposed system. In the near-field applications, different spot geometries at the near-field target plane can be achieved. In the far-field applications, the power matching of the MLEDA is used to improve far-field uniformity of spot array. Moreover, the predefined-geometry arrangement spot array can be realized in both near and far fields. Two design examples with full field of view (FOV) projection of 80° and 100° are provided to validate the proposed method. Overall, the proposed system offers a promising solution for various applications requiring target identification or 3D calibration.

Neural Free‐Viewpoint Relighting for Glossy Indirect Illumination

AbstractPrecomputed Radiance Transfer (PRT) remains an attractive solution for real‐time rendering of complex light transport effects such as glossy global illumination. After precomputation, we can relight the scene with new environment maps while changing viewpoint in real‐time. However, practical PRT methods are usually limited to low‐frequency spherical harmonic lighting. All‐frequency techniques using wavelets are promising but have so far had little practical impact. The curse of dimensionality and much higher data requirements have typically limited them to relighting with fixed view or only direct lighting with triple product integrals. In this paper, we demonstrate a hybrid neural‐wavelet PRT solution to high‐frequency indirect illumination, including glossy reflection, for relighting with changing view. Specifically, we seek to represent the light transport function in the Haar wavelet basis. For global illumination, we learn the wavelet transport using a small multi‐layer perceptron (MLP) applied to a feature field as a function of spatial location and wavelet index, with reflected direction and material parameters being other MLP inputs. We optimize/learn the feature field (compactly represented by a tensor decomposition) and MLP parameters from multiple images of the scene under different lighting and viewing conditions. We demonstrate real‐time (512 x 512 at 24 FPS, 800 x 600 at 13 FPS) precomputed rendering of challenging scenes involving view‐dependent reflections and even caustics.

VOLUMETRIC LIGHT TRANSFER IN FUNCTIONALLY DEFINED SCENES

In computer graphics, the main goals are high realism and rendering speed. In terms of realism, physically correct rendering is important, which calculates the transmission of light from the source to the observer. A method of modeling global illumination is presented, which takes into account light coming directly from sources, as well as light redirected through other parts of the scene. This allows you to create various optical effects, such as the flow of color from one surface to another, indirect lighting and caustics. The photon display method is designed for functionally defined scenes using graphics processors, thereby achieving an interactive rendering mode. The aim of the work is to implement a method for displaying photons in interactive rendering mode using graphics processors. It was necessary to find a compromise between physical accuracy and the speed of displaying scenes. As the number of parallel computing cores increases, traditional programming models become less efficient. GPUs based on the parallel processing model are being expanded by adding improved support for general-purpose computing to their cores. CUDA is a multicore computing platform. GPU-enabled hardware provides low-cost access to many computations. The method presented in this paper extends the photon-mapping algorithm for the CUDA platform to volumetric photon mapping. The beam brightness estimate used to efficiently calculate the density is recovered with the correct physical parameters. Photons are traced and bandwidth is selected. Performance increases due to calculations on multiple cores and the use of large compacted blocks.

Combined volume and surface rendering with global illumination caching

We present a combined volume and surface rendering technique with global illumination caching. Our approach uses volumetric path tracing to compute the global illumination volume and local shading models for rendering the isosurface. By joining both visualization approaches, we have enhanced the display and illumination of the surfaces while preserving physically realistic illumination of the participating media. To achieve real-time performance and avoid recomputing the image when the camera view changes, we compute the global illumination volume incrementally and defer the projection to a later step. We evaluated our technique by comparing different local shading models for isosurface rendering with the result of full volumetric path tracing and with the non-caching variant of our technique. Results show that the caching and non-caching variants perform comparably well, while the caching variant has the added benefit of being camera-view-independent. Additionally, we show that our approach emphasizes the surfaces within volumes better than volumetric path tracing.

Reconstruction of radiance field with neural network for real-time rendering

Fast global illumination methods has been one of the most exiting area of rendering because it could solve many industrial problems of achieving balance between realistic rendering and stable frame rate. So this study employs a neural network to learn a radiance field from the distribution of a particular group of objects by reconstructing a 3D radiance field using spherical harmonics and neural networks. The network learned the estimated radiance distribution in space by voxel data and creating a hash grid containing information from spherical harmonics. This research also provided a new simplified light transition function that emulated the behavior of light in a specific scene, as global illumination could be broken down into a sequence of the light transport progress. It may be possible to see how the light changes throughout a 3D environment by computing the global GI using this multiple-ray bounce model.

Importance-Based Ray Strategies for Dynamic Diffuse Global Illumination

In this paper, we propose a first and efficient ray allocation technique for Dynamic Diffuse Global Illumination (DDGI) using Multiple Importance Sampling (MIS). Our technique, IS-DDGI, extends DDGI by incorporating a set of importance-based ray strategies that analyze, allocate, and manage ray resources on the GPU. We combine these strategies with an adaptive historical and temporal frame-to-frame analysis for an effective reuse of information and a set of GPU-based optimizations for speeding up ray allocation and reducing memory bandwidth. Our IS-DDGI achieves similar visual quality to DDGI with a speedup of 1.27x to 2.47x in total DDGI time and 3.29x to 6.64x in probes ray tracing time over previous technique [Majercik et al. 2021]. Most speedup of IS-DDGI comes from probes ray tracing speedup.

Deep polyp image enhancement using region of interest with paired supervision

Endoscopic medical imaging in complex curved intestinal structures are prone to uneven illumination, low contrast and lack of texture information. These problems may lead to diagnostic challenges. This paper described the first supervised deep learning based image fusion framework to enable the polyp region highlight through a global image enhancement and a local region of interest (ROI) with paired supervision. Firstly, we conducted a dual attention based network in global image enhancement. The Detail Attention Maps was used to preserve more image details and the Luminance Attention Maps was used to adjust the global illumination of the image. Secondly, we adopted the advanced polyp segmentation network ACSNet to obtain the accurate mask image of lesion region in local ROI acquisition. Finally, a new image fusion strategy was proposed to realize the local enhancement effect of polyp image. Experimental results show that our method can highlight the local details of the lesion area better and reach the optimal comprehensive performance with comparing with 16 traditional and state-of-the-art enhancement algorithms. And 8 doctors and 12 medical students were asked to evaluate our method for assisting clinical diagnosis and treatment effectively. Furthermore, the first paired image dataset LHI was constructed, which will be made available as an open source to research communities.