Bidirectional Path Tracing Research Articles

Impact of Multi-Scattered LiDAR Returns in Fog.

In the context of autonomous driving, the augmentation of existing data through simulations provides an elegant solution to the challenge of capturing the full range of adverse weather conditions in training datasets. However, existing physics-based augmentation models typically rely on single scattering approximations to predict light propagation under unfavorable conditions, such as fog. This can prevent the reproduction of important signal characteristics encountered in a real-world environment. Consequently, in this work, Monte Carlo simulations are employed to assess the relevance of multiple-scattered light to the detected LiDAR signal in different types of fog, with scattering phase functions calculated from Mie theory considering real particle size distributions. Bidirectional path tracing is used within the self-developed GPU-accelerated Monte Carlo software to compensate for the unfavorable photon statistics associated with the limited detection aperture of the LiDAR geometry. To validate the Monte Carlo software, an analytical solution of the radiative transfer equation for the time-resolved radiance in terms of scattering orders is derived, thereby providing an explicit representation of the double-scattered contributions. The results of the simulations demonstrate that the shape of the detected signal can be significantly impacted by multiple-scattered light, depending on LiDAR geometry and visibility. In particular, double-scattered light can dominate the overall signal at low visibilities. This indicates that considering higher scattering orders is essential for improving AI-based perception models.

Hypothesis Testing for Progressive Kernel Estimation and VCM Framework.

Identifying an appropriate radius for unbiased kernel estimation is crucial for the efficiency of radiance estimation. However, determining both the radius and unbiasedness still faces big challenges. In this paper, we first propose a statistical model of photon samples and associated contributions for progressive kernel estimation, under which the kernel estimation is unbiased if the null hypothesis of this statistical model stands. Then, we present a method to decide whether to reject the null hypothesis about the statistical population (i.e., photon samples) by the F-test in the Analysis of Variance. Hereby, we implement a progressive photon mapping (PPM) algorithm, wherein the kernel radius is determined by this hypothesis test for unbiased radiance estimation. Second, we propose VCM+, a reinforcement of Vertex Connection and Merging (VCM), and derive its theoretically unbiased formulation. VCM+ combines hypothesis testing-based PPM with bidirectional path tracing (BDPT) via multiple importance sampling (MIS), wherein our kernel radius can leverage the contributions from PPM and BDPT. We test our new algorithms, improved PPM and VCM+, on diverse scenarios with different lighting settings. The experimental results demonstrate that our method can alleviate light leaks and visual blur artifacts of prior radiance estimate algorithms. We also evaluate the asymptotic performance of our approach and observe an overall improvement over the baseline in all testing scenarios.

Proxy Tracing: Unbiased Reciprocal Estimation for Optimized Sampling in BDPT

Robust light transport algorithms, particularly bidirectional path tracing (BDPT), face significant challenges when dealing with specular or highly glossy involved paths. BDPT constructs the full path by connecting sub-paths traced individually from the light source and camera. However, it remains difficult to sample by connecting vertices on specular and glossy surfaces with narrow-lobed BSDF, as it poses severe constraints on sampling in the feasible direction. To address this issue, we propose a novel approach, called proxy sampling , that enables efficient sub-path connection of these challenging paths. When a low-contribution specular/glossy connection occurs, we drop out the problematic neighboring vertex next to this specular/glossy vertex from the original path, then retrace an alternative sub-path as a proxy to complement this incomplete path. This newly constructed complete path ensures that the connection adheres to the constraint of the narrow lobe within the BSDF of the specular/glossy surface. Unbiased reciprocal estimation is the key to our method to obtain a probability density function (PDF) reciprocal to ensure unbiased rendering. We derive the reciprocal estimation method and provide an efficiency-optimized setting for efficient sampling and connection. Our method provides a robust tool for substituting problematic paths with favorable alternatives while ensuring unbiasedness. We validate this approach in the probabilistic connections BDPT for addressing specular-involved difficult paths. Experimental results have proved the effectiveness and efficiency of our approach, showcasing high-performance rendering capabilities across diverse settings.

Visualization of Solar Radiation Using Three-Dimensional Computer Graphics Technologies

Currently, the prediction, calculation and rationing of solar radiation is a serious problem that has a wide impact on the spheres of human activity. Creating and maintaining a comfortable living environment, as well as solving the world's energy problems, are the fundamental criteria that determine its relevance. This article discusses the possibility of using three-dimensional graphics technologies to calculate insolation. The paper considers an algorithm for calculating the four main components of the visualization of global solar radiation: direct beam radiation, diffuse radiation, reflected beam radiation and reflected diffuse radiation. The freely distributed open source software Blender 3D is used as a tool for working with computer graphics. In the course of the study, calculations based on the principle of bidirectional path tracing, which is used in the visualization algorithms of the Cycles graphics engine, are presented. The use of the technology of transferring complex data to the raster image format allows you to create a separate texture map for each component of the simulated lighting. Based on the obtained texture maps, it is possible to calculate the global solar radiation for all created or imported 3D models in the three-dimensional space of Blender 3D.

Warped-Area Reparameterization of Differential Path Integrals

Physics-based differentiable rendering is becoming increasingly crucial for tasks in inverse rendering and machine learning pipelines. To address discontinuities caused by geometric boundaries and occlusion, two classes of methods have been proposed: 1) the edge-sampling methods that directly sample light paths at the scene discontinuity boundaries, which require nontrivial data structures and precomputation to select the edges, and 2) the reparameterization methods that avoid discontinuity sampling but are currently limited to hemispherical integrals and unidirectional path tracing. We introduce a new mathematical formulation that enjoys the benefits of both classes of methods. Unlike previous reparameterization work that focused on hemispherical integral, we derive the reparameterization in the path space. As a result, to estimate derivatives using our formulation, we can apply advanced Monte Carlo rendering methods, such as bidirectional path tracing, while avoiding explicit sampling of discontinuity boundaries. We show differentiable rendering and inverse rendering results to demonstrate the effectiveness of our method.

Efficient Hardware Acceleration of Robust Volumetric Light Transport Simulation

AbstractEfficiently simulating the full range of light effects in arbitrary input scenes that contain participating media is a difficult task. Unified points, beams and paths (UPBP) is an algorithm capable of capturing a wide range of media effects, by combining bidirectional path tracing (BPT) and photon density estimation (PDE) with multiple importance sampling (MIS). A computationally expensive task of UPBP is the MIS weight computation, performed each time a light path is formed. We derive an efficient algorithm to compute the MIS weights for UPBP, which improves over previous work by eliminating the need to iterate over the path vertices. We achieve this by maintaining recursive quantities as subpaths are generated, from which the subpath weights can be computed. In this way, the full path weight can be computed by only using the data cached at the two vertices at the ends of the subpaths. Furthermore, a costly part of PDE is the search for nearby photon points and beams. Previous work has shown that a spatial data structure for photon mapping can be implemented using the hardware‐accelerated bounding volume hierarchy of NVIDIA's RTX GPUs. We show that the same technique can be applied to different types of volumetric PDE and compare the performance of these data structures with the state of the art. Finally, using our new algorithm and data structures we fully implement UPBP on the GPU which we, to the best of our knowledge, are the first to do so.

Bi-directional Monte-Carlo modelling of solar-induced chlorophyll fluorescence images for 3D vegetation canopies in the DART model

Remote sensing (RS) of solar-induced chlorophyll fluorescence (SIF) has a great potential for monitoring plant photosynthetic activity. Radiative transfer models (RTM) are essential to better interpret and extract information from SIF signals. DART is one of the most comprehensive and accurate 3D RTMs. Its standard mode DART-FT simulates SIF using a discrete ordinates method but is not adapted to large landscapes due to computational constraints. DART-Lux, the new mode based on a bi-directional path tracing algorithm, greatly improves DART computational efficiency for simulating images. This paper presents the theory of a novel SIF modelling algorithm in DART-Lux. We verified its accuracy with DART-FT and the SCOPE model for three types of canopies: turbid medium, maize field and forest. DART-Lux closely matches DART-FT (relative difference < 2%) with much better computational efficiency depending on the scene complexity, number of spectral bands and needed accuracy. For example, simulation time is reduced by a factor of ≈48, and memory usage by ≈50 for a maize field at 1 cm resolution. It allowed to simulate SIF images of large scenes as the 3×3km2 Ripperdan agricultural site that DART-FT could not simulate. The new SIF modelling algorithm opens new horizons for RS studies of large and complex landscapes. It is available as part of released DART versions (v1152 onwards) (https://dart.omp.eu/).

Light subpath reservoir for interactive ray-traced global illumination

In this paper, we propose a novel approach for sampling indirect illumination based on bidirectional path tracing and world space resampling reservoir. We cache light subpaths with high contribution in a world space grid, allowing for sampling at arbitrary vertices along the eye path. In order to find optimal samples, we continuously update the reservoirs stored in each cell according to the resampling principles. Experiments have shown that our algorithm achieves noticeable noise reduction in scenes lit by indirect light compared with path tracing, at the cost of additional time and memory consumption, while still maintaining interactive frame rates. In some special cases, our method even shows better results than a plain bidirectional path tracer, in terms of both image quality and performance.

Efficient Differentiation of Pixel Reconstruction Filters for Path-Space Differentiable Rendering

Pixel reconstruction filters play an important role in physics-based rendering and have been thoroughly studied. In physics-based differentiable rendering, however, the proper treatment of pixel filters remains largely under-explored. We present a new technique to efficiently differentiate pixel reconstruction filters based on the path-space formulation. Specifically, we formulate the pixel boundary integral that models discontinuities in pixel filters and introduce new antithetic sampling methods that support differentiable path sampling methods, such as adjoint particle tracing and bidirectional path tracing. We demonstrate both the need and efficacy of antithetic sampling when estimating this integral, and we evaluate its effectiveness across several differentiable- and inverse-rendering settings.

SPCBPT

Bidirectional path tracing (BDPT) can be accelerated by selecting appropriate light sub-paths for connection. However, existing algorithms need to perform frequent distribution reconstruction and have expensive overhead. We present a novel approach, SPCBPT, for probabilistic connections that constructs the light selection distribution in sub-path space. Our approach bins the sub-paths into multiple subspaces and keeps the sub-paths in the same subspace of low discrepancy, wherein the light sub-paths can be selected by a subspace-based two-stage sampling method, i.e., first sampling the light subspace and then resampling the light sub-paths within this subspace. The subspace-based distribution is free of reconstruction and provides efficient light selection at a very low cost. We also propose a method that considers the Multiple Importance Sampling (MIS) term in the light selection and thus obtain an MIS-aware distribution that can minimize the upper bound of variance of the combined estimator. Prior methods typically omit this MIS weights term. We evaluate our algorithm using various benchmarks, and the results show that our approach has superior performance and can significantly reduce the noise compared with the state-of-the-art method.

Efficiency-aware multiple importance sampling for bidirectional rendering algorithms

Multiple importance sampling (MIS) is an indispensable tool in light-transport simulation. It enables robust Monte Carlo integration by combining samples from several techniques. However, it is well understood that such a combination is not always more efficient than using a single sampling technique. Thus a major criticism of complex combined estimators, such as bidirectional path tracing, is that they can be significantly less efficient on common scenes than simpler algorithms like forward path tracing. We propose a general method to improve MIS efficiency: By cheaply estimating the efficiencies of various technique and sample-count combinations, we can pick the best one. The key ingredient is a numerically robust and efficient scheme that uses the samples of one MIS combination to compute the efficiency of multiple other combinations. For example, we can run forward path tracing and use its samples to decide which subset of VCM to enable, and at what sampling rates. The sample count for each technique can be controlled per-pixel or globally. Applied to VCM, our approach enables robust rendering of complex scenes with caustics, without compromising efficiency on simpler scenes.

Towards practical physical-optics rendering

Physical light transport (PLT) algorithms can represent the wave nature of light globally in a scene, and are consistent with Maxwell's theory of electromagnetism. As such, they are able to reproduce the wave-interference and diffraction effects of real physical optics. However, the recent works that have proposed PLT are too expensive to apply to real-world scenes with complex geometry and materials. To address this problem, we propose a novel framework for physical light transport based on several key ideas that actually makes PLT practical for complex scenes. First, we restrict the spatial coherence shape of light to an anisotropic Gaussian and justify this restriction with general arguments based on entropy. This restriction serves to simplify the rest of the derivations, without practical loss of generality. To describe partially-coherent light, we present new rendering primitives that generalize the radiometric radiance and irradiance, and are based on the well-known Stokes parameters. We are able to represent light of arbitrary spectral content and states of polarization, and with any coherence volume and anisotropy. We also present the wave BSDF to accurately render diffractions and wave-interference effects. Furthermore, we present an approach to importance sample this wave BSDF to facilitate bi-directional path tracing, which has been previously impossible. We show good agreement with state-of-the-art methods, but unlike them we are able to render complex scenes where all the materials are new, coherence-aware physical optics materials, and with performance approaching that of "classical" rendering methods.

DART-Lux: An unbiased and rapid Monte Carlo radiative transfer method for simulating remote sensing images

Accurate and efficient simulation of remote sensing images is increasingly needed in order to better exploit remote sensing observations and to better design remote sensing missions. DART (Discrete Anisotropic Radiative Transfer), developed since 1992 based on the discrete ordinates method (i.e., standard mode DART-FT), is one of the most accurate and comprehensive 3D radiative transfer models to simulate the radiative budget and remote sensing observations of urban and natural landscapes. Recently, a new method, called DART-Lux, was integrated into DART model to address the requirements of massive remote sensing data simulation for large-scale and complex landscapes. It is developed based on efficient Monte Carlo light transport algorithms (i.e., bidirectional path tracing) and on DART model framework. DART-Lux can accurately and rapidly simulate the bidirectional reflectance factor (BRF) and spectral images of arbitrary landscapes. This paper presents its theory, implementation, and evaluation. Its accuracy, efficiency and advantages are also discussed. The comparison with standard DART-FT in a variety of scenarios shows that DART-Lux is consistent with DART-FT (relative differences <1%) with simulation time and memory reduced by a hundredfold. DART-Lux is already part of the DART version freely available for scientists (https://dart.omp.eu).

Comprehensive LiDAR simulation with efficient physically-based DART-Lux model (I): Theory, novelty, and consistency validation

Light Detection And Ranging (LiDAR) remote sensing is increasingly needed to assess the 3D architecture of Earth's surface. Physically-based LiDAR radiative transfer (RT) models are essential tools for interpreting LiDAR signals, designing LiDAR systems, and validating information retrieval methods. Discrete Anisotropic Radiative Transfer (DART) is one of the most accurate and comprehensive 3D RT models that simulate LiDAR signals of urban and natural landscapes. Its physical modeling relies on a forward Monte Carlo mode optimized by a ray-tracking technique, also called DART-RC (Ray Carlo) mode. However, DART-RC is not adapted to simulate massive LiDAR signals of large landscapes due to its constraints of high memory demand and long computational time. Therefore, we developed a novel computationally efficient LiDAR modeling method based on a new DART modeling mode called DART-Lux. It simulates LiDAR signal by adapting the bidirectional path tracing algorithm of DART-Lux to the time and power measurements and by implementing the LiDAR instrument and multiple product outputs in DART-Lux. We verified the accuracy of DART-Lux for LiDAR modeling using DART-RC as a reference for several case studies with different LiDAR configurations (i.e., single-pulse waveform, multi-pulse point cloud, multi-pulse photon counting, with and without solar signal) on realistic scenes from the RAMI experiment. Results stress that i) DART-Lux is consistent with DART-RC, for example, R2 = 1 and rRMSE = 0.21% for the waveform of a forest simulated with a huge number of rays; ii) DART-Lux converges faster than DART-RC: its processing time is usually about half that of DART-RC, and over ten times smaller if the solar signal is simulated; iii) DART-Lux memory usage can be a hundred times less than DART-RC. Also, several sensitivity studies with various sensor configurations and solar directions illustrate the usefulness of DART-Lux for impact studies. This new DART-Lux LiDAR model opens promising perspectives for large-scale LiDAR applications with 3D modeling. It is already part of the official DART version freely available to scientists (https://dart.omp.eu).

Fast diffraction pathfinding for dynamic sound propagation

In the context of geometric acoustic simulation, one of the more perceptually important yet difficult to simulate acoustic effects is diffraction, a phenomenon that allows sound to propagate around obstructions and corners. A significant bottleneck in real-time simulation of diffraction is the enumeration of high-order diffraction propagation paths in scenes with complex geometry (e.g. highly tessellated surfaces). To this end, we present a dynamic geometric diffraction approach that consists of an extensive mesh preprocessing pipeline and complementary runtime algorithm. The preprocessing module identifies a small subset of edges that are important for diffraction using a novel silhouette edge detection heuristic. It also extends these edges with planar diffraction geometry and precomputes a graph data structure encoding the visibility between the edges. The runtime module uses bidirectional path tracing against the diffraction geometry to probabilistically explore potential paths between sources and listeners, then evaluates the intensities for these paths using the Uniform Theory of Diffraction. It uses the edge visibility graph and the A* pathfinding algorithm to robustly and efficiently find additional high-order diffraction paths. We demonstrate how this technique can simulate 10th-order diffraction up to 568 times faster than the previous state of the art, and can efficiently handle large scenes with both high geometric complexity and high numbers of sources.

Correlation‐Aware Multiple Importance Sampling for Bidirectional Rendering Algorithms

AbstractCombining diverse sampling techniques via multiple importance sampling (MIS) is key to achieving robustness in modern Monte Carlo light transport simulation. Many such methods additionally employ correlated path sampling to boost efficiency. Photon mapping, bidirectional path tracing, and path‐reuse algorithms construct sets of paths that share a common prefix. This correlation is ignored by classical MIS heuristics, which can result in poor technique combination and noisy images. We propose a practical and robust solution to that problem. Our idea is to incorporate correlation knowledge into the balance heuristic, based on known path densities that are already required for MIS. This correlation‐aware heuristic can achieve considerably lower error than the balance heuristic, while avoiding computational and memory overhead.

Accurate and fast simulation of remote sensing images at top of atmosphere with DART-Lux

DART model is one of the most comprehensive and accurate radiative transfer (RT) models to simulate remotely sensed signals in the Earth-atmosphere system. Its standard RT modelling mode, called DART-FT, relies on the discrete ordinates method. Its recently developed Monte Carlo mode using an unbiased bidirectional path tracing method, called DART-Lux, increases hundredfold DART efficiency to simulate images. Since DART-Lux does not simulate yet atmospheric RT, a hybrid method has been designed to accurately and fast simulate remote sensing images at top of atmosphere (TOA). It couples the atmospheric RT modelling of DART-FT with the very efficient Earth surface RT modelling of DART-Lux. For that, a new sky light modelling, an innovative BRF camera modelling and an adapted radiative coupling framework have been designed. The efficiency and accuracy of this hybrid method have been validated using DART-FT as a reference. Here, we present this validation for a realistic forest stand. Simulation time is reduced by a factor of 750 with a relative difference of solar plane reflectance factor smaller than 1%. This hybrid method opens new perspectives for the use of 3D modelling in remote sensing applications. It is already part of the DART version freely available for scientists (https://dart.omp.eu).

Calculation of Noise Components for Bidirectional Path Tracing with Photon Maps

The classic Monte-Carlo ray tracing is a powerful method which allows to simulate virtually all effects in ray optics, but it may be inadmissibly slow for many cases, such as calculation of images seen by a lens or pin-hole camera. In this cases another stochastic method is more efficient such as the bi-directional ray Monte-Carlo tracing with photon maps (BDPM). The level of noise i.e. the r.m.s. (root mean square) of pixel luminance calculated in one iteration of the method, depends on various parameters of the method, such as the number of light and camera paths, radius of integration sphere etc. so it is desirable to be able to predict this dependence to choose optimal parameters of the method. It was shown that this r.m.s is a sum of 3 functions scaled by reverse number of camera and light rays. These functions themselves are independent of the number of rays, so knowing them one can predict the noise for any number of rays and thus find the optimal one. These functions are a sort of correlations and their calculation from ray tracing is not a trivial problem. In this paper we describe a practical method of calculation and demonstrate the usage of its results for the choice of ray number.

Path tracing estimators for refractive radiative transfer

Rendering radiative transfer through media with a heterogeneous refractive index is challenging because the continuous refractive index variations result in light traveling along curved paths. Existing algorithms are based on photon mapping techniques, and thus are biased and result in strong artifacts. On the other hand, existing unbiased methods such as path tracing and bidirectional path tracing cannot be used in their current form to simulate media with a heterogeneous refractive index. We change this state of affairs by deriving unbiased path tracing estimators for this problem. Starting from the refractive radiative transfer equation (RRTE), we derive a path-integral formulation, which we use to generalize path tracing with next-event estimation and bidirectional path tracing to the heterogeneous refractive index setting. We then develop an optimization approach based on fast analytic derivative computations to produce the point-to-point connections required by these path tracing algorithms. We propose several acceleration techniques to handle complex scenes (surfaces and volumes) that include participating media with heterogeneous refractive fields. We use our algorithms to simulate a variety of scenes combining heterogeneous refraction and scattering, as well as tissue imaging techniques based on ultrasonic virtual waveguides and lenses. Our algorithms and publicly-available implementation can be used to characterize imaging systems such as refractive index microscopy, schlieren imaging, and acousto-optic imaging, and can facilitate the development of inverse rendering techniques for related applications.

Path differential-informed stratified MCMC and adaptive forward path sampling

Markov Chain Monte Carlo (MCMC) rendering is extensively studied, yet it remains largely unused in practice. We propose solutions to several practicability issues, opening up path space MCMC to become an adaptive sampling framework around established Monte Carlo (MC) techniques. We address non-uniform image quality by deriving an analytic target function for imagespace sample stratification. The function is based on a novel connection between variance and path differentials, allowing analytic variance estimates for MC samples, with potential uses in other adaptive algorithms outside MCMC. We simplify these estimates down to simple expressions using only quantities known in any MC renderer. We also address the issue that most existing MCMC renderers rely on bi-directional path tracing and reciprocal transport, which can be too costly and/or too complex in practice. Instead, we apply our theoretical framework to optimize an adaptive MCMC algorithm that only uses forward path construction. Notably, we construct our algorithm by adapting (with minimal changes) a full-featured path tracer into a single-path state space Markov Chain, bridging another gap between MCMC and existing MC techniques.