Light Transport Model Research Articles

Advancing Photorealism Enhancement Using Convolutional Neural Networks (CNNs)

Photorealism has been the defining goal of computer graphics for half a century. A look at even the most sophisticated real-time games will quickly reveal that photorealism has not been achieved. An ineffable difference in the appearance of simulation and reality remains. In recent years, a complementary set of techniques has been developed in computer vision and machine learning. These techniques, based on deep learning, convolutional networks, and adversarial training, bypass physical modeling of geometric layout, material appearance, and light transport. Instead, images are synthesized by convolutional networks trained on large datasets. These techniques have been used to synthesize representative images from a given domain to convert semantic label maps to photographic images and to attempt to bridge the appearance gap between synthetic and real images. The images are enhanced by a convolutional network that leverages intermediate representations produced by conventional rendering pipelines. The network is trained via a novel adversarial objective, which provides strong supervision at multiple perceptual levels.

Stochastic Gauss-Newton method for estimating absorption and scattering in optical tomography with the Monte Carlo method for light transport.

Image reconstruction in optical tomography in the so-called transport regime, where the diffusion approximation is not valid, requires modeling of light transport using the radiative transfer equation. In this work, we approach this problem by utilizing the Monte Carlo method for light transport. In this work, we propose a methodology for absolute imaging of absorption and scattering in this regime utilizing a Monte Carlo method for light transport. The image reconstruction problem is formulated as a minimization problem that is solved using a stochastic Gauss-Newton method. In the construction of the Jacobian matrix for scattering, a perturbation approximation for Monte Carlo is utilized. The approach is evaluated with numerical simulations using an adaptive approach where the number of photon packets is adjusted during the iterations, and with different fixed numbers of photon packets. The simulations show that the Monte Carlo method for light transport can be utilized in the absolute imaging problem of optical tomography and that the absorption and scattering parameters can be estimated simultaneously with good accuracy.

Correlations of light scattering properties in human skin with the person's age assessed using a non-invasive technique.

We analyze the influence of a person's age on the thicknesses and reduced scattering coefficients of the epidermis and dermis in visible part of the spectrum. Their values were assessed using a non-invasive technique which combines pulsed photothermal radiometry and diffuse reflectance spectroscopy with Monte Carlo modeling of light transport in a four-layer model of skin. The analysis is affected by the strong influences of the melanin content on the reduced scattering coefficient of the epidermis, a epi, and blood content in the case of dermis (a der). Separating their contributions reveals a significant decrease of a der with the person's age at an average rate of -0.25 mm-1 per decade, while the contribution of blood in the papillary dermis amounts to 1.0 mm-1%-1. Meanwhile, no influence of the person's age was found on a epi and the thicknesses of the epidermis or dermis.

Deep learning acceleration of iterative model-based light fluence correction for photoacoustic tomography

Photoacoustic tomography (PAT) is a promising imaging technique that can visualize the distribution of chromophores within biological tissue. However, the accuracy of PAT imaging is compromised by light fluence (LF), which hinders the quantification of light absorbers. Currently, model-based iterative methods are used for LF correction, but they require extensive computational resources due to repeated LF estimation based on differential light transport models. To improve LF correction efficiency, we propose to use Fourier neural operator (FNO), a neural network specially designed for estimating partial differential equations, to learn the forward projection of light transport in PAT. Trained using paired finite-element-based LF simulation data, our FNO model replaces the traditional computational heavy LF estimator during iterative correction, such that the correction procedure is considerably accelerated. Simulation and experimental results demonstrate that our method achieves comparable LF correction quality to traditional iterative methods while reducing the correction time by over 30 times.

Cohesive framework for non-line-of-sight imaging based on Dirac notation.

The non-line-of-sight (NLOS) imaging field encompasses both experimental and computational frameworks that focus on imaging elements that are out of the direct line-of-sight, for example, imaging elements that are around a corner. Current NLOS imaging methods offer a compromise between accuracy and reconstruction time as experimental setups have become more reliable, faster, and more accurate. However, all these imaging methods implement different assumptions and light transport models that are only valid under particular circumstances. This paper lays down the foundation for a cohesive theoretical framework which provides insights about the limitations and virtues of existing approaches in a rigorous mathematical manner. In particular, we adopt Dirac notation and concepts borrowed from quantum mechanics to define a set of simple equations that enable: i) the derivation of other NLOS imaging methods from such single equation (we provide examples of the three most used frameworks in NLOS imaging: back-propagation, phasor fields, and f-k migration); ii) the demonstration that the Rayleigh-Sommerfeld diffraction operator is the propagation operator for wave-based imaging methods; and iii) the demonstration that back-propagation and wave-based imaging formulations are equivalent since, as we show, propagation operators are unitary. We expect that our proposed framework will deepen our understanding of the NLOS field and expand its utility in practical cases by providing a cohesive intuition on how to image complex NLOS scenes independently of the underlying reconstruction method.

Towards a more accurate light transport model for non-line-of-sight imaging.

Non-line-of-sight (NLOS) imaging systems involve the measurement of an optical signal at a diffuse surface. A forward model encodes the physics of these measurements mathematically and can be inverted to generate a reconstruction of the hidden scene. Some existing NLOS imaging techniques rely on illuminating the diffuse surface and measuring the photon time of flight (ToF) of multi-bounce light paths. Alternatively, some methods depend on measuring high-frequency variations caused by shadows cast by occluders in the hidden scene. While forward models for ToF-NLOS and Shadow-NLOS have been developed separately, there has been limited work on unifying these two imaging modalities. Dove et al introduced a unified mathematical framework capable of modeling both imaging techniques [Opt. Express27, 18016 (2019)10.1364/OE.27.018016]. The authors utilize this general forward model, known as the two frequency spatial Wigner distribution (TFSWD), to discuss the implications of reconstruction resolution for combining the two modalities but only when the occluder geometry is known a priori. In this work, we develop a graphical representation of the TFSWD forward model and apply it to novel experimental setups with potential applications in NLOS imaging. Furthermore, we use this unified framework to explore the potential of combining these two imaging modalities in situations where the occluder geometry is not known in advance.

Non-Invasive Hemoglobin Assessment with NIR Imaging of Blood Vessels in Transmittance Geometry: Monte Carlo and Experimental Evaluation

Non-invasive methods for determining blood hemoglobin (Hb) concentration are urgently needed to avoid the painful and time-consuming process of invasive venous blood sampling. Many such methods rely on assessing the average attenuation of light over a tissue area where hemoglobin is the dominant chromophore, without separating those areas corresponding to vessels and bloodless tissue. In this study, we investigate whether it is possible to determine hemoglobin levels in the blood by assessing the changes in light intensity when passing through large vessels in comparison to adjacent tissues, using this as a Hb level predictor. Using Monte Carlo light transport modeling, we evaluate the accuracy of determining hemoglobin levels via light intensity contrast and vessel widths estimated in the transmittance illumination geometry and estimate the influence of physiologically significant parameters such as vessel depth, dermis vascularization, and melanin content in the epidermis on the blood Hb prediction error. The results show that physiological variations in tissue parameters limit the mean absolute error of this method to ~15 g/L for blood Hb levels varying in the 60–160 g/L range, which finding is also supported by experimental data obtained for volunteers with different total blood Hb levels that have been determined invasively. We believe the application of new approaches to the non-invasive assessment of Hb levels will lead to the creation of reliable and accurate devices that are applicable in point-of-care and clinical practice.

Rendering the Bluish Appearance of Snow: When Light Transmission Matters.

Material appearance is largely determined by complex light attenuation processes. The distinct bluish colorations that can be observed when light is transmitted through snow are among the most striking outcomes of these processes. In this article, we present a method for the predictive rendering of this phenomenon taking into account the variability of snow's physical and morphological characteristics. To achieve that, we employ an approach centered on the effective use of spectral transmittance data obtained using a first-principles light transport model for snow. The suitability of the proposed method to rendering applications is illustrated through the synthesis of images depicting the bluish appearance of snow under different illumination conditions.

Quantitative photoacoustic tomography: modeling and inverse problems.

Quantitative photoacoustic tomography (QPAT) exploits the photoacoustic effect with the aim of estimating images of clinically relevant quantities related to the tissue's optical absorption. The technique has two aspects: an acoustic part, where the initial acoustic pressure distribution is estimated from measured photoacoustic time-series, and an optical part, where the distributions of the optical parameters are estimated from the initial pressure. Our study is focused on the optical part. In particular, computational modeling of light propagation (forward problem) and numerical solution methodologies of the image reconstruction (inverse problem) are discussed. The commonly used mathematical models of how light and sound propagate in biological tissue are reviewed. A short overview of how the acoustic inverse problem is usually treated is given. The optical inverse problem and methods for its solution are reviewed. In addition, some limitations of real-life measurements and their effect on the inverse problems are discussed. An overview of QPAT with a focus on the optical part was given. Computational modeling and inverse problems of QPAT were addressed, and some key challenges were discussed. Furthermore, the developments for tackling these problems were reviewed. Although modeling of light transport is well-understood and there is a well-developed framework of inverse mathematics for approaching the inverse problem of QPAT, there are still challenges in taking these methodologies to practice. Modeling and inverse problems of QPAT together were discussed. The scope was limited to the optical part, and the acoustic aspects were discussed only to the extent that they relate to the optical aspect.

Fundamental ray tracing project based on OpenGL with acceleration

In 3D computer graphics, ray tracing is an essential technique for modeling light transport for use in a wide variety of rendering algorithms for generating digital images. And it is also an important subject for computer science students who want to discover more in Computer Graphics and image rendering. In this project the author implemented a fundamental light tracing system based on C++ in visual studio. It is able to render a delicate image with an input of a nnf model file, and output a ppm file, as a final assignment of a student computer graphics research program. This paper is divided into two chapters. The first chapter is about some fundamental theories that are critical to light tracing. The second chapter is about detailed design and implementation process of the program. And the final chapter is about some actual problems and effective solutions.

QOCT-Net: A Physics-Informed Neural Network for Intravascular Optical Coherence Tomography Attenuation Imaging.

Intravascular optical coherence tomography (IVOCT) provides high-resolution, depth-resolved images of coronary arterial microstructure by acquiring backscattered light. Quantitative attenuation imaging is important for accurate characterization of tissue components and identification of vulnerable plaques. In this work, we proposed a deep learning method for IVOCT attenuation imaging based on the multiple scattering model of light transport. A physics-informed deep network named Quantitative OCT Network (QOCT-Net) was designed to recover pixel-level optical attenuation coefficients directly from standard IVOCT B-scan images. The network was trained and tested on simulation and in vivo datasets. Results showed superior attenuation coefficient estimates both visually and based on quantitative image metrics. The structural similarity, energy error depth and peak signal-to-noise ratio are improved by at least 7%, 5% and 12.4%, respectively, compared with the state-of-the-art non-learning methods. This method potentially enables high-precision quantitative imaging for tissue characterization and vulnerable plaque identification.

A Hyperspectral Space of Skin Tones for Inverse Rendering of Biophysical Skin Properties

AbstractWe present a method for estimating the main properties of human skin, leveraging a hyperspectral dataset of skin tones synthetically generated through a biophysical layered skin model and Monte Carlo light transport simulations. Our approach learns the mapping between the skin parameters and diffuse skin reflectance in such space through an encoder‐decoder network. We assess the performance of RGB and spectral reflectance up to 1 μm, allowing the model to retrieve visible and near‐infrared. Instead of restricting the parameters to values in the ranges reported in medical literature, we allow the model to exceed such ranges to gain expressiveness to recover outliers like beard, eyebrows, rushes and other imperfections. The continuity of our albedo space allows to recover smooth textures of skin properties, enabling reflectance manipulations by meaningful edits of the skin properties. The space is robust under different illumination conditions, and presents high spectral similarity with the current largest datasets of spectral measurements of real human skin while expanding its gamut.

Evaluation of light collection from highly scattering media using wavelength-shifting fibers

Opaque scintillators are designed to have a short scattering length such that scintillation photons are probabilistically confined to a small region of space about their origin. The benefit of this feature is that information on the interaction event topology can be recorded with greater fidelity than in traditional highly transparent media with sensors at large distances from the light production region. Opaque scintillator detectors rely on wavelength-shifting fibers to extract the scintillation light; however, the efficiency of light collection has not yet been directly measured in experiment. We measured the efficiency of light collection as a function of the optical parameters of an opaque liquid and the distance from the origin of the light to the fiber. We use the experimental data to validate a Monte Carlo model of light transport and collection and discuss a simple diffusion model that reproduces the results of Monte Carlo simulation with high fidelity. This combination of validated models has the potential for use in predictions of performance in various designs of future opaque scintillator detectors such as LiquidO.

Extended angular-spectrum modeling (EASM) of light energy transport in scattering media.

The exact modeling of light transport in scattering media is critical in biological imaging, free-space communication, and phosphor-converted lighting. Angular spectrum is proved to be a fast and effective approach to reconstructing the wavefront dynamics during the propagation in scattering media, however, finding it difficult in acquiring the wavefront and energy change simultaneously. Besides, conventional methods for energy tracing, such as the Monte Carlo method, are inefficient in speed and hard to simulate the wavefront change. Here, we propose an extended angular-spectrum modeling (EASM) approach using tenuous scattering approximate solutions to obtain a time-efficient and accurate method for reconstruction of energy and wavefront dynamics in various scattering media. The generality of our method is numerically simulated and experimentally verified with a set of scattering media with different properties. EASM has a time advantage under the guarantee of calculation accuracy, especially when calculating several thickness changes after the calculation model is established. Furthermore, multi-layered media can also be simulated by EASM with a good precision. The results suggest that EASM performs certain computations more efficiently than the conventional method and thus provides an effective and flexible calculation tool for scattering media.

RGB camera-based simultaneous measurements of percutaneous arterial oxygen saturation, tissue oxygen saturation, pulse rate, and respiratory rate.

We propose a method to perform simultaneous measurements of percutaneous arterial oxygen saturation (SpO 2), tissue oxygen saturation (StO 2), pulse rate (PR), and respiratory rate (RR) in real-time, using a digital red–green–blue (RGB) camera. Concentrations of oxygenated hemoglobin (C HbO), deoxygenated hemoglobin (C HbR), total hemoglobin (C HbT), and StO 2 were estimated from videos of the human face using a method based on a tissue-like light transport model of the skin. The photoplethysmogram (PPG) signals are extracted from the temporal fluctuations in C HbO, C HbR, and C HbT using a finite impulse response (FIR) filter (low and high cut-off frequencies of 0.7 and 3 Hz, respectively). The PR is calculated from the PPG signal for C HbT. The ratio of pulse wave amplitude for C HbO and that for C HbR are associated with the reference value of SpO 2 measured by a commercially available pulse oximeter, which provides an empirical formula to estimate SpO 2 from videos. The respiration-dependent oscillation in C HbT was extracted from another FIR filter (low and high cut-off frequencies of 0.05 and 0.5 Hz, respectively) and used to calculate the RR. In vivo experiments with human volunteers while varying the fraction of inspired oxygen were performed to evaluate the comparability of the proposed method with commercially available devices. The Bland–Altman analysis showed that the mean bias for PR, RR, SpO 2, and StO 2 were -1.4 (bpm), -1.2(rpm), 0.5 (%), and -3.0 (%), respectively. The precisions for PR, RR, Sp O 2, and StO 2 were ±3.1 (bpm), ±3.5 (rpm), ±4.3 (%), and ±4.8 (%), respectively. The resulting precision and RMSE for StO 2 were pretty close to the clinical accuracy requirement. The accuracy of the RR is considered a little less accurate than clinical requirements. This is the first demonstration of a low-cost RGB camera-based method for contactless simultaneous measurements of the heart rate, percutaneous arterial oxygen saturation, and tissue oxygen saturation in real-time.

Estimation of porcine pancreas optical properties in the 600–1100 nm wavelength range for light-based therapies

This work reports the optical properties of porcine pancreatic tissue in the broad wavelength range of 600–1100 nm. Absorption and reduced scattering coefficients (µa and µs′) of the ex vivo pancreas were obtained by means of Time-domain Diffuse Optical Spectroscopy. We have investigated different experimental conditions—including compression, repositioning, spatial sampling, temporal stability—the effect of the freezing procedure (fresh vs frozen-thawed pancreas), and finally inter-sample variability. Good repeatability under different experimental conditions was obtained (median coefficient of variation less than 8% and ~ 16% for µa and µs′, respectively). Freezing–thawing the samples caused an irreversible threefold reduction of µs′ and no effect on µa. The absorption and reduced scattering spectra averaged over different samples were in the range of 0.12–0.74 cm−1 and 12–21 cm−1 with an inter-sample variation of ~ 10% and ~ 40% for µa and µs′, respectively. The calculated effective transport coefficient (µeff) for fresh pancreatic tissue shows that regions between 800–900 nm and 1050–1100 nm are similar and offer the lowest tissue attenuation in the considered range (i.e., µeff ranging from 2.4 to 2.7 cm−1). These data, describing specific light-pancreas interactions in the therapeutic optical window for the first time, provide pivotal information for planning of light-based thermotherapies (e.g., laser ablation) and instruction of light transport models for biophotonic applications involving this organ.

Meshless Monte Carlo radiation transfer method for curved geometries using signed distance functions.

.Significance: Monte Carlo radiation transfer (MCRT) is the gold standard for modeling light transport in turbid media. Typical MCRT models use voxels or meshes to approximate experimental geometry. A voxel-based geometry does not allow for the precise modeling of smooth curved surfaces, such as may be found in biological systems or food and drink packaging. Mesh-based geometry allows arbitrary complex shapes with smooth curved surfaces to be modeled. However, mesh-based models also suffer from issues such as the computational cost of generating meshes and inaccuracies in how meshes handle reflections and refractions.Aim: We present our algorithm, which we term signedMCRT (sMCRT), a geometry-based method that uses signed distance functions (SDF) to represent the geometry of the model. SDFs are capable of modeling smooth curved surfaces precisely while also modeling complex geometries.Approach: We show that using SDFs to represent the problem’s geometry is more precise than voxel and mesh-based methods.Results: sMCRT is validated against theoretical expressions, and voxel and mesh-based MCRT codes. We show that sMCRT can precisely model arbitrary complex geometries such as microvascular vessel network using SDFs. In comparison with the current state-of-the-art in MCRT methods specifically for curved surfaces, sMCRT is more precise for cases where the geometry can be defined using combinations of shapes.Conclusions: We believe that SDF-based MCRT models are a complementary method to voxel and mesh models in terms of being able to model complex geometries and accurately treat curved surfaces, with a focus on precise simulation of reflections and refractions. sMCRT is publicly available at https://github.com/lewisfish/signedMCRT.

History of Monte Carlo modeling of light transport in tissues using mcml.c.

.The Monte Carlo simulation called mcml.c was written and shared on-line since 1992. This perspective summarizes the contributions by the people involved in the development of mcml.c, and work by others extending the code.

Development and validation of quantitative optical index of skin blood content.

.SignificanceWe present an approach to estimate with simple instrumentation the amount of red blood cells in the skin microvasculature, designated as parameter . Variations of parameter are shown to reflect local changes in the quantity of skin red blood cells during a venous occlusion challenge.AimTo validate a simple algebraic model of light transport in skin using the Monte Carlo method and to develop a measure of the red blood cell content in skin microvessels using the Monte Carlo predictions; to guide the development of an instrument to measure experimentally variations of the amount of red blood cells in the skin.ApproachMonte Carlo simulations were carried out in a multilayer model of the skin to compute remitted light intensities as a function of distance from the illumination locus for different values of the skin blood content. The simulation results were used to compute parameter and its variations with local skin blood content. An experimental setup was developed to measure parameter in human volunteers in whom skin blood content of the forearm increased during temporary interruption of the venous outflow.ResultsIn the simulations, parameter was in baseline conditions, and it increased in near proportion with the blood content of the skin layers. Measuring the diffusely reflected light intensity 0.5 to 1.2 mm away from the illumination locus was optimal to detect appreciable changes of the reflected light intensity as skin blood content was altered. Parameter measured experimentally on the human forearm was in baseline conditions it increased at a rate of when venous outflow was temporarily interrupted.ConclusionParameter derived experimentally with a two-wavelength diffuse reflectometer can be used to measure local variations of the amount of red blood cells in skin microvessels.

Combined Non-Invasive Optical Oximeter and Flowmeter with Basic Metrological Equipment

Optical non-invasive diagnostic methods and equipment are used today in many medical disciplines. However, there is still no generally accepted and unifying engineering theory of such systems. Today, they are developed most empirically and do not always have the desired effectiveness in clinics. Among reasons for their insufficient clinical efficiency, we can claim the limited set of measured parameters, the poorly substantiated technical design parameters, and the lack of metrological certification, which all together lead to large uncertainties and inaccuracies in diagnostic data. The purpose of this study is to develop a new instrument for non-invasive optical oximetry by means of substantiating and creating amore informative tissue oximeter with an enhanced number of measured parameters and equipped with the basic metrological tools—imitational measures. The combination of two related optical diagnostic techniques—a tissue oximetry, including a cerebral one, and a fluctuation flowmetry on a single hardware platform—was used. Theoretical modeling of light transport in tissues was applied to substantiate the main technical design parameters of the device. For each measuring channel, relevant imitation measures for metrological verification and adjustment have been proposed. Some common principles for the operation of such equipment are described in the article, as well.