Path Tracing Research Articles

A Clinical User Study Investigating the Benefits of Adaptive Volumetric Illumination Sampling.

Accurate and fast understanding of the patient's anatomy is crucial in surgical decision making and particularly important in visceral surgery. Sophisticated visualization techniques such as 3D Volume Rendering can aid the surgeon and potentially lead to a benefit for the patient. Recently, we proposed a novel volume rendering technique called Adaptive Volumetric Illumination Sampling (AVIS) that can generate realistic lighting in real-time, even for high resolution images and volumes but without introducing additional image noise. In order to evaluate this new technique, we conducted a randomized, three-period crossover study comparing AVIS to conventional Direct Volume Rendering (DVR) and Path Tracing (PT). CT datasets from 12 patients were evaluated by 10 visceral surgeons who were either senior physicians or experienced specialists. The time needed for answering clinically relevant questions as well as the correctness of the answers were analyzed for each visualization technique. In addition to that, the perceived workload during these tasks was assessed for each technique, respectively. The results of the study indicate that AVIS has an advantage in terms of both time efficiency and most aspects of the perceived workload, while the average correctness of the given answers was very similar for all three methods. In contrast to that, Path Tracing seems to show particularly high values for mental demand and frustration. We plan to repeat a similar study with a larger participant group to consolidate the results.

Simulation of Optical Tactile Sensors Supporting Slip and Rotation Using Path Tracing and IMPM

Simulation of Optical Tactile Sensors Supporting Slip and Rotation Using Path Tracing and IMPM

ChartLine: Automatic Detection and Tracing of Curves in Scientific Line Charts Using Spatial-Sequence Feature Pyramid Network.

Line charts are prevalent in scientific documents and commercial data visualization, serving as essential tools for conveying data trends. Automatic detection and tracing of line paths in these charts is crucial for downstream tasks such as data extraction, chart quality assessment, plagiarism detection, and visual question answering. However, line graphs present unique challenges due to their complex backgrounds and diverse curve styles, including solid, dashed, and dotted lines. Existing curve detection algorithms struggle to address these challenges effectively. In this paper, we propose ChartLine, a novel network designed for detecting and tracing curves in line graphs. Our approach integrates a Spatial-Sequence Attention Feature Pyramid Network (SSA-FPN) in both the encoder and decoder to capture rich hierarchical representations of curve structures and boundary features. The model incorporates a Spatial-Sequence Fusion (SSF) module and a Channel Multi-Head Attention (CMA) module to enhance intra-class consistency and inter-class distinction. We evaluate ChartLine on four line chart datasets and compare its performance against state-of-the-art curve detection, edge detection, and semantic segmentation methods. Extensive experiments demonstrate that our method significantly outperforms existing algorithms, achieving an F-measure of 94% on a synthetic dataset.

CrystalNet: Texture‐Aware Neural Refraction Baking for Global Illumination

AbstractNeural rendering bakes global illumination and other computationally costly effects into the weights of a neural network, allowing to efficiently synthesize photorealistic images without relying on path tracing. In neural rendering approaches, G‐buffers obtained from rasterization through direct rendering provide information regarding the scene such as position, normal, and textures to the neural network, achieving accurate and stable rendering quality in real‐time. However, due to the use of G‐buffers, existing methods struggle to accurately render transparency and refraction effects, as G‐buffers do not capture any ray information from multiple light ray bounces. This limitation results in blurriness, distortions, and loss of detail in rendered images that contain transparency and refraction, and is particularly notable in scenes with refracted objects that have high‐frequency textures. In this work, we propose a neural network architecture to encode critical rendering information, including texture coordinates from refracted rays, and enable reconstruction of high‐frequency textures in areas with refraction. Our approach is able to achieve accurate refraction rendering in challenging scenes with a diversity of overlapping transparent objects. Experimental results demonstrate that our method can interactively render high quality refraction effects with global illumination, unlike existing neural rendering approaches. Our code can be found at https://github.com/ziyangz5/CrystalNet

Sorghum segmentation and leaf counting using in silico trained deep neural model

AbstractThis paper introduces a novel deep neural model for segmenting and tracking the number of leaves in sorghum plants in phenotyping facilities. Our algorithm inputs a sequence of images of a sorghum plant and outputs the segmented images and the number of leaves. The key novelty of our approach is in training the deep neural model. Manual annotations are tedious, and we have developed a procedural three‐dimensional (3D) sorghum model that provides detailed geometry and texture to generate photorealistic 3D models. The overall shape of the sorghum leaf geometry is determined by its skeleton, and it is detailed by a procedural model that varies its curvature, width, length, and overall shape. The color is determined by using a Monte Carlo path tracer. We mimic the illumination of the phenotyping facility and use reflectance and transmittance on sorghum surfaces to determine the color of the leaves. The 3D procedural model allows us to generate photorealistic and segmented images that we use to train a deep neural model. Our segmentation provides a mean intersection over union score of 0.51, resulting in leaf counting accuracy within the 90% confidence interval for the human counts.

Efficient participating media rendering with differentiable regularization

Highly scattering media, such as milk, skin, and clouds, are common in the real world. Rendering participating media is challenging, especially for high-order scattering dominant media, because the light may undergo a large number of scattering events before leaving the surface. Monte Carlo-based methods typically require a long time to produce noise-free results. Based on the observation that low-albedo media contain less noise than high-albedo media, we propose reducing the variance of the rendered results using differentiable regularization. We first render an image with low-albedo participating media together with the gradient with respect to the albedo, and then predict the final rendered image with a low-albedo image and gradient image via a novel prediction function. To achieve high quality, we also consider the gradients of neighboring frames to provide a noise-free gradient image. Ultimately, our method can produce results with much less overall error than equal-time path tracing methods.

Multi-Feature Fusion in Graph Convolutional Networks for Data Network Propagation Path Tracing

With the rapid development of information technology, the complexity of data networks is increasing, especially in electric power systems, where data security and privacy protection are of great importance. Throughout the entire distribution process of the supply chain, it is crucial to closely monitor the propagation paths and dynamics of electrical data to ensure security and quickly initiate comprehensive traceability investigations if any data tampering is detected. This research addresses the challenges of data network complexity and its impact on the security of power systems by proposing an innovative data network propagation path tracing model, which is constructed based on graph convolutional networks (GCNs) and the BERT model. Firstly, propagation trees are constructed based on the propagation structure, and the key attributes of data nodes are extracted and screened. Then, GCNs are utilized to learn the representation of node features with different attribute feature combinations in the propagation path graph, while the Bidirectional Encoder Representations from Transformers (BERT) model is employed to capture the deep semantic features of the original text content. The core of this research is to effectively integrate these two feature representations, namely the structural features obtained by GCNs and the semantic features obtained by the BERT model, in order to enhance the ability of the model to recognize the data propagation path. The experimental results demonstrate that this model performs well in power data propagation and tracing tasks, and the data recognition accuracy reaches 92.5%, which is significantly better than the existing schemes. This achievement not only improves the power system’s ability to cope with data security threats but also provides strong support for protecting data transmission security and privacy.

Hardware information flow tracking based on lightweight path awareness

Vulnerabilities and Trojans in hardware design may cause sensitive data to be leaked and tampered. Information flow tracking technology can effectively verify the confidentiality and integrity of hardware design. Currently, this technology mainly analyzes the reachability of information flow and lacks fine-grained analysis of information flow paths. It is difficult to find structural defects in information flow paths and malicious sensitive information processes in hardware design. To solve above problem, we propose Path-aware Dynamic Information Flow Tracking (PDIFT) technology, which performs taint tracking and path tracking while sensitive information is propagated. It analyzes the propagation of sensitive information in hardware design with fine-grained taint label propagation logic and inserts path label propagation logic only on basic blocks divided by branch nodes, which greatly simplifies the path tracing overhead compared to the full node sequence tracing on the path. Experiments have shown that compared to CellIFT, PDIFT has a 12.1% increase in static analysis time and a 0.1% increase in dynamic validation time. The average instrumentation area cost of each basic block has increased by 16.4 um2. In terms of detection capability, PDIFT makes up for the limitation of false negatives in traditional taint tracking technology through joint analysis of path labels and taint labels, then detect problems such as insufficient iterations of encryption components and malicious processing of important assets, thereby improving the accuracy of hardware security verification.

Light Path Guided Culling for Hybrid Real-Time Path Tracing

Rendering visually convincing images requires realistic lighting. Path tracing has long been used in offline rendering to produce photorealistic images. While recent hardware advancements allow ray tracing methods to be employed in real-time renderers, they come with a significant performance and memory impact. Real-time path tracing remains a challenge. We present light path guided culling (LiPaC), a novel culling algorithm for ray tracing that achieves almost optimal culling results by considering the number of light paths encountered by objects. In addition, we describe a hybrid path tracing pipeline using LiPaC to render large and highly dynamic scenes in real-time on the current generation of consumer hardware.

Impact of Multi-Scattered LiDAR Returns in Fog.

In the context of autonomous driving, the augmentation of existing data through simulations provides an elegant solution to the challenge of capturing the full range of adverse weather conditions in training datasets. However, existing physics-based augmentation models typically rely on single scattering approximations to predict light propagation under unfavorable conditions, such as fog. This can prevent the reproduction of important signal characteristics encountered in a real-world environment. Consequently, in this work, Monte Carlo simulations are employed to assess the relevance of multiple-scattered light to the detected LiDAR signal in different types of fog, with scattering phase functions calculated from Mie theory considering real particle size distributions. Bidirectional path tracing is used within the self-developed GPU-accelerated Monte Carlo software to compensate for the unfavorable photon statistics associated with the limited detection aperture of the LiDAR geometry. To validate the Monte Carlo software, an analytical solution of the radiative transfer equation for the time-resolved radiance in terms of scattering orders is derived, thereby providing an explicit representation of the double-scattered contributions. The results of the simulations demonstrate that the shape of the detected signal can be significantly impacted by multiple-scattered light, depending on LiDAR geometry and visibility. In particular, double-scattered light can dominate the overall signal at low visibilities. This indicates that considering higher scattering orders is essential for improving AI-based perception models.

Hypothesis Testing for Progressive Kernel Estimation and VCM Framework.

Identifying an appropriate radius for unbiased kernel estimation is crucial for the efficiency of radiance estimation. However, determining both the radius and unbiasedness still faces big challenges. In this paper, we first propose a statistical model of photon samples and associated contributions for progressive kernel estimation, under which the kernel estimation is unbiased if the null hypothesis of this statistical model stands. Then, we present a method to decide whether to reject the null hypothesis about the statistical population (i.e., photon samples) by the F-test in the Analysis of Variance. Hereby, we implement a progressive photon mapping (PPM) algorithm, wherein the kernel radius is determined by this hypothesis test for unbiased radiance estimation. Second, we propose VCM+, a reinforcement of Vertex Connection and Merging (VCM), and derive its theoretically unbiased formulation. VCM+ combines hypothesis testing-based PPM with bidirectional path tracing (BDPT) via multiple importance sampling (MIS), wherein our kernel radius can leverage the contributions from PPM and BDPT. We test our new algorithms, improved PPM and VCM+, on diverse scenarios with different lighting settings. The experimental results demonstrate that our method can alleviate light leaks and visual blur artifacts of prior radiance estimate algorithms. We also evaluate the asymptotic performance of our approach and observe an overall improvement over the baseline in all testing scenarios.

Neural SSS: Lightweight Object Appearance Representation

AbstractWe present a method for capturing the BSSRDF (bidirectional scattering‐surface reflectance distribution function) of arbitrary geometry with a neural network. We demonstrate how a compact neural network can represent the full 8‐dimensional light transport within an object including heterogeneous scattering. We develop an efficient rendering method using importance sampling that is able to render complex translucent objects under arbitrary lighting. Our method can also leverage the common planar half‐space assumption, which allows it to represent one BSSRDF model that can be used across a variety of geometries. Our results demonstrate that we can render heterogeneous translucent objects under arbitrary lighting and obtain results that match the reference rendered using volumetric path tracing.

Residual path integrals for re‐rendering

AbstractConventional rendering techniques are primarily designed and optimized for single‐frame rendering. In practical applications, such as scene editing and animation rendering, users frequently encounter scenes where only a small portion is modified between consecutive frames. In this paper, we develop a novel approach to incremental re‐rendering of scenes with dynamic objects, where only a small part of a scene moves from one frame to the next. We formulate the difference (or residual) in the image between two frames as a (correlated) light‐transport integral which we call the residual path integral. Efficient numerical solution of this integral then involves (1) devising importance sampling strategies to focus on paths with non‐zero residual‐transport contributions and (2) choosing appropriate mappings between the native path spaces of the two frames. We introduce a set of path importance sampling strategies that trace from the moving object(s) which are the sources of residual energy. We explore path mapping strategies that generalize those from gradient‐domain path tracing to our importance sampling techniques specially for dynamic scenes. Additionally, our formulation can be applied to material editing as a simpler special case. We demonstrate speed‐ups over previous correlated sampling of path differences and over rendering the new frame independently. Our formulation brings new insights into the re‐rendering problem and paves the way for devising new types of sampling techniques and path mappings with different trade‐offs.

A Free-Space Diffraction BSDF

Free-space diffractions are an optical phenomenon where light appears to "bend" around the geometric edges and corners of scene objects. In this paper we present an efficient method to simulate such effects. We derive an edge-based formulation of Fraunhofer diffraction, which is well suited to the common (triangular) geometric meshes used in computer graphics. Our method dynamically constructs a free-space diffraction BSDF by considering the geometry around the intersection point of a ray of light with an object, and we present an importance sampling strategy for these BSDFs. Our method is unique in requiring only ray tracing to produce free-space diffractions, works with general meshes, requires no geometry preprocessing, and is designed to work with path tracers with a linear rendering equation. We show that we are able to reproduce accurate diffraction lobes, and, in contrast to any existing method, are able to handle complex, real-world geometry. This work serves to connect free-space diffractions to the efficient path tracing tools from computer graphics.

Proxy Tracing: Unbiased Reciprocal Estimation for Optimized Sampling in BDPT

Robust light transport algorithms, particularly bidirectional path tracing (BDPT), face significant challenges when dealing with specular or highly glossy involved paths. BDPT constructs the full path by connecting sub-paths traced individually from the light source and camera. However, it remains difficult to sample by connecting vertices on specular and glossy surfaces with narrow-lobed BSDF, as it poses severe constraints on sampling in the feasible direction. To address this issue, we propose a novel approach, called proxy sampling , that enables efficient sub-path connection of these challenging paths. When a low-contribution specular/glossy connection occurs, we drop out the problematic neighboring vertex next to this specular/glossy vertex from the original path, then retrace an alternative sub-path as a proxy to complement this incomplete path. This newly constructed complete path ensures that the connection adheres to the constraint of the narrow lobe within the BSDF of the specular/glossy surface. Unbiased reciprocal estimation is the key to our method to obtain a probability density function (PDF) reciprocal to ensure unbiased rendering. We derive the reciprocal estimation method and provide an efficiency-optimized setting for efficient sampling and connection. Our method provides a robust tool for substituting problematic paths with favorable alternatives while ensuring unbiasedness. We validate this approach in the probabilistic connections BDPT for addressing specular-involved difficult paths. Experimental results have proved the effectiveness and efficiency of our approach, showcasing high-performance rendering capabilities across diverse settings.

Stylized Rendering as a Function of Expectation

We propose a generalization of the rendering equation that captures both the realistic light transport of physically-based rendering (PBR) and a subset of non-photorealistic rendering (NPR) stylizations in a principled manner. The proposed formulation is based on the key observation that both classical transport and certain NPR stylizations can be modeled as a function of expectation. Given this observation, we generalize the recursive integrals of the rendering equation to recursive functions of expectation. As estimating functions of expectation can be challenging, especially recursive ones, we provide a toolkit for unbiased and biased estimation comprising prior work, general strategies, and a novel build-your-own strategy for constructing more complex unbiased estimators from simpler unbiased estimators. We then use this toolkit to construct a complete estimator for the proposed recursive formulation, and implement a sampling algorithm that is both conceptually simple and leverages many of the components of an ordinary path tracer. To demonstrate the practicality of the proposed method we showcase how it captures several existing stylizations like color mapping, cel shading, and cross-hatching, fuses NPR and PBR visuals, and allows us to explore visuals that were previously challenging under existing formulations.

Spontaneous path tracing in task-irrelevant mazes: Spatial affordances trigger dynamic visual routines.

Given a maze (e.g., in a book of puzzles), you might solve it by drawing out paths with your pencil. But even without a pencil, you might naturally find yourself mentally tracing along various paths. This "mental path tracing" may intuitively seem to depend on your (overt, conscious, voluntary) goal of wanting to get out of the maze, but might it also occur spontaneously-as a result of simply seeing the maze, via a kind of dynamic visual routine? Here, observers simply had to compare the visual properties of two probes presented in a maze. The maze itself was entirely task irrelevant, but we predicted that simply seeing the maze's visual structure would "afford" incidental mental path tracing (à la Gibson). Across four experiments, observers were slower to compare probes that were further from each other along the paths, even when controlling for lower level properties (such as the probes' brute linear separation, ignoring the maze "walls"). These results also generalized beyond mazes to other unfamiliar displays with task-irrelevant circular obstacles. This novel combination of two prominent themes from our field-affordances and visual routines-suggests that at least some visual routines may not require voluntary goals; instead, they may operate in an automatic (incidental, stimulus-driven) fashion, as a part of visual processing itself. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

TraM‐NeRF: Tracing Mirror and Near‐Perfect Specular Reflections Through Neural Radiance Fields

AbstractImplicit representations like neural radiance fields (NeRF) showed impressive results for photorealistic rendering of complex scenes with fine details. However, ideal or near‐perfectly specular reflecting objects such as mirrors, which are often encountered in various indoor scenes, impose ambiguities and inconsistencies in the representation of the re‐constructed scene leading to severe artifacts in the synthesized renderings. In this paper, we present a novel reflection tracing method tailored for the involved volume rendering within NeRF that takes these mirror‐like objects into account while avoiding the cost of straightforward but expensive extensions through standard path tracing. By explicitly modelling the reflection behaviour using physically plausible materials and estimating the reflected radiance with Monte‐Carlo methods within the volume rendering formulation, we derive efficient strategies for importance sampling and the transmittance computation along rays from only few samples. We show that our novel method enables the training of consistent representations of such challenging scenes and achieves superior results in comparison to previous state‐of‐the‐art approaches.

Application of Riemannian Seismic Ray Path Tracing in Salt Dome Prospecting

Within the geophysical exploration utilising seismic methods, it is well known that if the explored distances are much greater than the wavelength of the seismic waves with which the exploration is carried out, the ray approach of the wave theory can be used. In this way, when the rays travel through an inhomogeneous medium, they follow curved trajectories, which is imperative to determine the geological features that produce reflection and refraction phenomena. In this paper, a simple algorithm for the calculation of the trajectory of a seismic beam through an inhomogeneous stratum is presented. For this, the construction of a pseudo-Riemannian metric is required from the function of P-wave velocities of the geological stratum. Thus, the problem is inverted because instead of finding the curved trajectory of the seismic beam in a background with a Euclidean metric, it is proposed that the beam follows a geodesic of a curved space-time specific to each stratum, becoming a simple and automatic process using the differential geometry apparatus. For the reader to gain insight into this tool, different geological setups from idealised ones up to a salt dome are presented.

Refined tri-directional path tracing with generated light portal

Refined tri-directional path tracing with generated light portal