Shading Point Research Articles

Real-Time Path Guiding Using Bounding Voxel Sampling

We propose a real-time path guiding method, Voxel Path Guiding (VXPG), that significantly improves fitting efficiency under limited sampling budget. Our key idea is to use a spatial irradiance voxel data structure across all shading points to guide the location of path vertices. For each frame, we first populate the voxel data structure with irradiance and geometry information. To sample from the data structure for a shading point, we need to select a voxel with high contribution to that point. To importance sample the voxels while taking visibility into consideration, we adapt techniques from offline many-lights rendering by clustering pairs of shading points and voxels. Finally, we unbiasedly sample within the selected voxel while taking the geometry inside into consideration. Our experiments show that VXPG achieves significantly lower perceptual error compared to other real-time path guiding and virtual point light methods under equal-time comparison. Furthermore, our method does not rely on temporal information, but can be used together with other temporal reuse sampling techniques such as ReSTIR to further improve sampling efficiency.

LightFormer: Light-Oriented Global Neural Rendering in Dynamic Scene

The generation of global illumination in real time has been a long-standing challenge in the graphics community, particularly in dynamic scenes with complex illumination. Recent neural rendering techniques have shown great promise by utilizing neural networks to represent the illumination of scenes and then decoding the final radiance. However, incorporating object parameters into the representation may limit their effectiveness in handling fully dynamic scenes. This work presents a neural rendering approach, dubbed LightFormer , that can generate realistic global illumination for fully dynamic scenes, including dynamic lighting, materials, cameras, and animated objects, in real time. Inspired by classic many-lights methods, the proposed approach focuses on the neural representation of light sources in the scene rather than the entire scene, leading to the overall better generalizability. The neural prediction is achieved by leveraging the virtual point lights and shading clues for each light. Specifically, two stages are explored. In the light encoding stage, each light generates a set of virtual point lights in the scene, which are then encoded into an implicit neural light representation, along with screen-space shading clues like visibility. In the light gathering stage, a pixel-light attention mechanism composites all light representations for each shading point. Given the geometry and material representation, in tandem with the composed light representations of all lights, a lightweight neural network predicts the final radiance. Experimental results demonstrate that the proposed LightFormer can yield reasonable and realistic global illumination in fully dynamic scenes with real-time performance.

Neural Global Illumination: Interactive Indirect Illumination Prediction under Dynamic Area Lights.

We propose neural global illumination, a novel method for fast rendering full global illumination in static scenes with dynamic viewpoint and area lighting. The key idea of our method is to utilize a deep rendering network to model the complex mapping from each shading point to global illumination. To efficiently learn the mapping, we propose a neural-network-friendly input representation including attributes of each shading point, viewpoint information, and a combinational lighting representation that enables high-quality fitting with a compact neural network. To synthesize high-frequency global illumination effects, we transform the low-dimension input to higher-dimension space by positional encoding and model the rendering network as a deep fully-connected network. Besides, we feed a screen-space neural buffer to our rendering network to share global information between objects in the screen-space to each shading point. We have demonstrated our neural global illumination method in rendering a wide variety of scenes exhibiting complex and all-frequency global illumination effects such as multiple-bounce glossy interreflection, color bleeding, and caustics.

Efficient Caustics Rendering via Spatial and Temporal Path Reuse

AbstractCaustics are complex optical effects caused by the light being concentrated in a small area due to reflection or refraction on surfaces with low roughness, typically under a sharp light source. Rendering caustic effects is challenging for Monte Carlo‐based approaches, due to the difficulties of sampling the specular paths. One effective solution is using the specular manifold to locate these valid specular paths. Unfortunately, it needs many iterations to find these paths, leading to a long rendering time. To address this issue, our key insight is that the specular paths tend to be similar for neighboring shading points. To this end, we propose to reuse the specular paths spatially. More specifically, we generate some specular path samples with a low sample rate and then reuse these specular path samples as the initialization for specular manifold walk among neighboring shading points. In this way, much fewer specular path‐searching iterations are performed, due to the efficient initialization close to the final solution. Furthermore, this reuse strategy can be extended for dynamic scenes in a temporal manner, such as light moving or specular geometry deformation. Our method outperforms current state‐of‐the‐art methods and can handle multiple bounces of light and various scenes.

Interactive VPL-based global illumination on the GPU using fuzzy clustering

Physically-based synthesis of high quality imagery, including global illumination light transport phenomena, results in a significant workload, which makes interactive rendering a very challenging task. We propose a VPL-based ray tracing approach that runs entirely in the GPU and achieves interactive frame rates while handling global illumination light transport phenomena. This approach is based on clustering both shading points and VPLs and computing visibility only among clusters’ representatives. A new massively parallel K-means clustering algorithm, enables efficient execution in the GPU. Rendering artifacts, that could result from the piecewise constant approximation of the VPLs/shading points visibility function introduced by the clustering, are smoothed away by resorting to an innovative approach based on fuzzy clustering and weighted interpolation of the visibility function. The effectiveness of the proposed approach is experimentally verified for a collection of scenes, with frame rates larger than 3 fps and up to 25 fps being demonstrated.

Recursive analytic spherical harmonics gradient for spherical lights

AbstractWhen rendering images using Spherical Harmonics (SH), the projection of a spherical function on the SH basis remains a computational challenge both for high‐frequency functions and for emission functions from complex light sources. Recent works investigate efficient SH projection of the light field coming from polygonal and spherical lights. To further reduce the rendering time, instead of computing the SH coefficients at each vertex of a mesh or at each fragment on an image, it has been shown, for polygonal area light, that computing both the SH coefficients and their spatial gradients on a grid covering the scene allows the efficient and accurate interpolation of these coefficients at each shaded point. In this paper, we develop analytical recursive formulae to compute the spatial gradients of SH coefficients for spherical light. This requires the efficient computation of the spatial gradients of the SH basis function that we also derive. Compared to existing method for polygonal light, our method is faster, requires less memory and scales better with respect to the SH band limit. We also show how to approximate polygonal lights using spherical lights to benefit from our derivations. To demonstrate the effectiveness of our proposal, we integrate our algorithm in a shading system able to render fully dynamic scenes with several hundreds of spherical lights in real time.

Meshlets and How to Shade Them: A Study on Texture‐Space Shading

AbstractCommonly used image‐space layouts of shading points, such as used in deferred shading, are strictly view‐dependent, which restricts efficient caching and temporal amortization. In contrast, texture‐space layouts can represent shading on all surface points and can be tailored to the needs of a particular application. However, the best grouping of shading points—which we call a shading unit—in texture space remains unclear. Choices of shading unit granularity (how many primitives or pixels per unit) and in shading unit parametrization (how to assign texture coordinates to shading points) lead to different outcomes in terms of final image quality, overshading cost, and memory consumption. Among the possible choices, shading units consisting of larger groups of scene primitives, so‐called meshlets, remain unexplored as of yet. In this paper, we introduce a taxonomy for analyzing existing texture‐space shading methods based on the group size and parametrization of shading units. Furthermore, we introduce a novel texture‐space layout strategy that operates on large shading units: the meshlet shading atlas. We experimentally demonstrate that the meshlet shading atlas outperforms previous approaches in terms of image quality, run‐time performance and temporal upsampling for a given number of fragment shader invocations. The meshlet shading atlas lends itself to work together with popular cluster‐based rendering of meshes with high geometric detail.

Novel accelerated Stochastic Progressive Photon Mapping rendering with neural network

Recently, deep learning-based approaches have led to dramatic improvements for Monte Carlo rendering at the low sampling rate. Most of these approaches are aimed at path tracing. However, they are not suitable for photon mapping. In this paper, we develop a novel accelerate stochastic progressive photon mapping approaches with neural network. First, our framework utilizes the particle-based rendering and focuses on photon density estimation. We train a neural network to predict a kernel function to aggregate photon contributions at shading point. Then we construct a estimation images with the prediction network. During experiments, we could find that there are spike pixels and noises in estimation images sometimes. So we present the improved denoising network to post-process the estimation images. Finally, we can obtain the high-quality reconstructions of complex global illumination effects like caustics with an order of magnitude fewer photons compared with previous photon mapping methods. Besides, our denoising network can reduce most multi-scale noises on both low-frequency and high-frequency areas while preserving more illumination details, especially caustics, compared with other state-of-the-art learning-based denoising methods.

Analytic spherical harmonic gradients for real-time rendering with many polygonal area lights

Recent work has developed analytic formulae for spherical harmonic (SH) coefficients from uniform polygonal lights, enabling near-field area lights to be included in Precomputed Radiance Transfer (PRT) systems, and in offline rendering. However, the method is inefficient since coefficients need to be recomputed at each vertex or shading point, for each light, even though the SH coefficients vary smoothly in space. The complexity scales linearly with the number of lights, making many-light rendering difficult. In this paper, we develop a novel analytic formula for the spatial gradients of the spherical harmonic coefficients for uniform polygonal area lights. The result is a significant generalization, involving the Reynolds transport theorem to reduce the problem to a boundary integral for which we derive a new analytic formula, showing how to reduce a key term to an earlier recurrence for SH coefficients. The implementation requires only minor additions to existing code for SH coefficients. The results also hold implications for recent efforts on differentiable rendering. We show that SH gradients enable very sparse spatial sampling, followed by accurate Hermite interpolation. This enables scaling PRT to hundreds of area lights with minimal overhead and real-time frame rates. Moreover, the SH gradient formula is a new mathematical result that potentially enables many other graphics applications.

Deep Kernel Density Estimation for Photon Mapping

AbstractRecently, deep learning‐based denoising approaches have led to dramatic improvements in low sample‐count Monte Carlo rendering. These approaches are aimed at path tracing, which is not ideal for simulating challenging light transport effects like caustics, where photon mapping is the method of choice. However, photon mapping requires very large numbers of traced photons to achieve high‐quality reconstructions. In this paper, we develop the first deep learning‐based method for particle‐based rendering, and specifically focus on photon density estimation, the core of all particle‐based methods. We train a novel deep neural network to predict a kernel function to aggregate photon contributions at shading points. Our network encodes individual photons into per‐photon features, aggregates them in the neighborhood of a shading point to construct a photon local context vector, and infers a kernel function from the per‐photon and photon local context features. This network is easy to incorporate in many previous photon mapping methods (by simply swapping the kernel density estimator) and can produce high‐quality reconstructions of complex global illumination effects like caustics with an order of magnitude fewer photons compared to previous photon mapping methods. Our approach largely reduces the required number of photons, significantly advancing the computational efficiency in photon mapping.

An Approach to Global Illumination Calculation Based on Hybrid Cone Tracing

For 3D geographic information systems (GIS) or video game systems, global illumination (GI) effect can greatly improve the understanding of the volumetric structure and the spatial relationships of objects in the scene. However, GI effect is computationally expensive. It requires not only complex visibility computations between arbitrary points but also the integration computation over a large number of directions. These two computations become extremely difficult tasks for the real-time or interactive algorithms of the dynamic scene. This paper proposes a lightweight GI calculation approach built upon a hybrid cone tracing algorithm with which to approximate the GI effect of the open world scene in real time. First, the 3D scene is divided into two types: complex meshes and height-field meshes. Second, the corresponding lightmaps of the two meshes are generated and mipmaped, respectively. GPU hardware acceleration technology is used to do the calculation efficiently. Finally, a hybrid cone tracing method is performed on the GPU to gather the indirect lighting information for each shaded point. The method first carries out the cone-scene intersection test and then carries out the light sampling and accumulation calculation. All tracing calculations are based on the hybrid lightmap representation. In addition, the experiment results show the effectiveness of our method. Compared with the 3D texture VCT method, memory consumption can be reduced by up to 80% in our experiment.

A task and data balanced distributed photon mapping method

Photon mapping is an important global illumination method of simulating some visual effects, such as caustic. However, photon mapping is difficult to be parallelized efficiently, due to unbalanced task distribution and memory management. In this paper, we present a combined data-parallel and task-parallel approach to improve performance across multiple rendering clients in a distributed system. The shading points (receivers) from pixels are divided into groups. We associate the photons with each receiver group by computing a tree cut of the photon hierarchy, thus both the data and the task are tightly coupled. Furthermore, we organize both the photon hierarchy and the receivers with Morton code for efficient tree construction. In the end, our method balances both the illumination task of receivers and related photons, resulting in better load balancing and higher performance without any quality degradation.

Importance Sampling of Many Lights with Adaptive Tree Splitting

We present a technique to importance sample large collections of lights (including mesh lights as collections of small emitters) in the context of Monte-Carlo path tracing. A bounding volume hierarchy over all emitters is traversed at each shading point using a single random number in a way that importance samples their predicted contribution. The tree aggregates energy, spatial and orientation information from the emitters to enable accurate prediction of the effect of a cluster of lights on any given shading point. We further improve the performance of the algorithm by forcing splitting until the importance of a cluster is sufficiently representative of its contents.

Efficient Energy-Compensated VPLs using Photon Splatting

Efficient light transport simulation is important for many applications in computer graphics. Many-light methods based on virtual point lights (VPLs) especially work well for scenes consist mainly of diffuse materials. Rendered images with VPLs, however, are known to suffer from energy loss due to clamping of the VPL contributions. Removing this clamping, unfortunately, introduces significant variance in images due to the singularities of VPLs. We propose an efficient and practical algorithm which compensates energy loss of VPLs using yet another efficient rendering method -- photon splatting. Recent unified path sampling frameworks show that VPLs are in fact photons. Based on this unified formulation, we propose to reuse VPLs as photons to compensate missing energy of clamped VPLs. This combination is very natural: the singularities of VPLs happen if a shading point is too close to VPLs, but such VPLs can also be seen as neighboring photons that are indeed useful for photon density estimation. We formulate this energy compensation process using multiple importance sampling to combine rendering with VPLs and photon splatting optimally. Our algorithm can be efficiently implemented on GPUs since rendering with VPLs and photon splatting are both suitable for GPUs. The results demonstrate that our method produces more accurate images in the same rendering time than the existing methods. For common use cases of rendering mostly diffuse scenes, our work leads to an efficient, accurate, and practical rendering algorithm based on VPLs.

Microfacet-based normal mapping for robust Monte Carlo path tracing

Normal mapping enhances the amount of visual detail of surfaces by using shading normals that deviate from the geometric normal. However, the resulting surface model is geometrically impossible and normal mapping is thus often considered a fundamentally flawed approach with unavoidable problems for Monte Carlo path tracing, such as asymmetry, back-facing normals, and energy loss arising from this incoherence. These problems are usually sidestepped in real-time renderers, but they cannot be fixed robustly in a path tracer: normal mapping breaks either the appearance (black fringes, energy loss) or the integrator (different forward and backward light transport); in practice, workarounds and tweaked normal maps are often required to hide artifacts. We present microfacet-based normal mapping, an alternative way of faking geometric details without corrupting the robustness of Monte Carlo path tracing. It takes the same input data as classic normal mapping and works with any input BRDF. Our idea is to construct a geometrically valid microfacet surface made of two facets per shading point: the one given by the normal map at the shading point and an additional facet that compensates for it such that the average normal of the microsurface equals the geometric normal. We derive the resulting microfacet BRDF and show that it mimics geometric detail in a plausible way, although it does not replicate the appearance of classic normal mapping. However, our microfacet-based normal mapping model is well-defined, symmetric, and energy conserving, and thus yields identical results with any path tracing algorithm (forward, backward, or bidirectional).

Vectorized point based global illumination on Intel MIC architecture

Point-based global illumination (PBGI) is a popular rendering method in visual special effects and motion picture productions. This rendering algorithm models the 3D scene as a dense point cloud, which acts as caching records for light transport simulation. Structured in a tree, this cache supports the image synthesis stage through massive adaptive tree cut searches, together with the projection of these cuts on the many receiver shading points i.e., unprojected pixels in 3D space, where visibility is solved with receiver-specific z-buffer. These two operations are both time consuming in this algorithm, but they can be formulated for efficient parallel execution, in particular regarding wide-SIMD hardware. During the PBGI tree traversal procedure, we introduce a single-receiver traversal scheme for incoherent receivers, a packet traversal scheme for coherent receivers, as well as logic for dynamically switching between these methods at run-time. During the per-receiver rasterization procedure, we propose three different vectorization strategies for near-, mid- and far-distance points separately. We conducted experiments on an Intel Many Integrated Core (MIC) architecture and report results on several scenes, showing that up to a 9 × speedup can be achieved when compared with non-vectorized execution during the traversal step, and nearly 2.5 × speedup during rasterization step without quality degradation.

Experimental investigation of the effect of partial shading on photovoltaic performance

Partial shading of a photovoltaic (PV) installation has an inconsistent impact on power production. This study investigates the effect of partial shading on PV performance. The experiments were carried out with a 90-W PV module under both variable and constant irradiations with shaded area increased from 0 to 80% to observe the effect of variable solar radiation at certain shading points. The effect of shading under irradiation levels from 300 to 800 W/m2 was investigated. At a 600 W/m2 irradiation level, the shading impact factor was 1.25 with a 25% shading, while at 75% shading the impact factor decreased to 0.86. Results also show that for every 100 W/m2 increase in irradiation level, the electrical power output was enhanced by 3.89, 3.37, 2.27, and 2.02 W at 0, 25, 50, and 75% shading, respectively. The efficiency level was increased by 0.29, 0.27, 0.25, and 0.22% at 0, 25, 50, and 75% shading, respectively. Increasing the shaded area by 10% causes a 12.41 W drop in power output and a 2.3% drop in electrical efficiency. Partial shading not only deteriorates the PV performance, but also causes long-term degradation of the module.

Texture space caching and reconstruction for ray tracing

We present a texture space caching and reconstruction system for Monte Carlo ray tracing. Our system gathers and filters shading on-demand, including querying secondary rays, directly within a filter footprint around the current shading point. We shade on local grids in texture space with primary visibility decoupled from shading. Unique filters can be applied per material, where any terms of the shader can be chosen to be included in each kernel. This is a departure from recent screen space image reconstruction techniques, which typically use a single, complex kernel with a set of large auxiliary guide images as input. We show a number of high-performance use cases for our system, including interactive denoising of Monte Carlo ray tracing with motion/defocus blur, spatial and temporal shading reuse, cached product importance sampling, and filters based on linear regression in texture space.

Anisotropic Ambient Volume Shading.

We present a novel method to compute anisotropic shading for direct volume rendering to improve the perception of the orientation and shape of surface-like structures. We determine the scale-aware anisotropy of a shading point by analyzing its ambient region. We sample adjacent points with similar scalar values to perform a principal component analysis by computing the eigenvectors and eigenvalues of the covariance matrix. In particular, we estimate the tangent directions, which serve as the tangent frame for anisotropic bidirectional reflectance distribution functions. Moreover, we exploit the ratio of the eigenvalues to measure the magnitude of the anisotropy at each shading point. Altogether, this allows us to model a data-driven, smooth transition from isotropic to strongly anisotropic volume shading. In this way, the shape of volumetric features can be enhanced significantly by aligning specular highlights along the principal direction of anisotropy. Our algorithm is independent of the transfer function, which allows us to compute all shading parameters once and store them with the data set. We integrated our method in a GPU-based volume renderer, which offers interactive control of the transfer function, light source positions, and viewpoint. Our results demonstrate the benefit of anisotropic shading for visualization to achieve data-driven local illumination for improved perception compared to isotropic shading.

Anisotropic density estimation for photon mapping

Photon mapping is a widely used technique for global illumination rendering. In the density estimation step of photon mapping, the indirect radiance at a shading point is estimated through a filtering process using nearby stored photons; an isotropic filtering kernel is usually used. However, using an isotropic kernel is not always the optimal choice, especially for cases when eye paths intersect with surfaces with anisotropic BRDFs. In this paper, we propose an anisotropic filtering kernel for density estimation to handle such anisotropic eye paths. The anisotropic filtering kernel is derived from the recently introduced anisotropic spherical Gaussian representation of BRDFs. Compared to conventional photon mapping, our method is able to reduce rendering errors with negligible additional cost when rendering scenes containing anisotropic BRDFs.