Global Illumination Research Articles

Learnable differencing center for nighttime depth perception

Depth completion is the task of recovering dense depth map from sparse ones, usually with the help of color images. Existing image guided methods perform well on daytime depth perception self-driving benchmarks, but struggle in nighttime scenarios with poor visibility and complex illumination. To address these challenges, we propose a simple yet effective learnable differencing center network (LDCNet). The key idea is to use recurrent inter-convolution differencing (RICD) and illumination affinitive intra-convolution differencing (IAICD) to enhance the nighttime color images and reduce the negative effects of the varying illumination, respectively. RICD explicitly estimates global illumination by differencing two convolutions with different kernels, treating the small-kernel-convolution feature as the center of the large-kernel-convolution feature in a new perspective. IAICD softly alleviates the local relative light intensity by differencing a single convolution, where the center is dynamically aggregated based on neighboring pixels and the estimated illumination map in the RICD. On both nighttime depth completion and depth estimation tasks, extensive experiments demonstrate the effectiveness of our LDCNet, reaching the state of the art.

Beyond ExaBricks: GPU Volume Path Tracing of AMR Data

AbstractAdaptive Mesh Refinement (AMR) is becoming a prevalent data representation for HPC, and thus also for scientific visualization. AMR data is usually cell centric (which imposes numerous challenges), complex, and generally hard to render. Recent work on GPU‐accelerated AMR rendering has made much progress towards real‐time volume and isosurface rendering of such data, but so far this work has focused exclusively on ray marching, with simple lighting models and without scattering events or global illumination. True high‐quality rendering requires a modified approach that is able to trace arbitrary incoherent paths; but this may not be a perfect fit for the types of data structures recently developed for ray marching. In this paper, we describe a novel approach to high‐quality path tracing of complex AMR data, with a specific focus on analyzing and comparing different data structures and algorithms to achieve this goal.

High-quality lateral monolayer-multilayer graphene junction formed by selective laser thinning for self-powered photodetection

The formation of an asymmetric junction is key to graphene-based photodetectors of high-sensitive photodetectability, because such a junction can not only facilitate the diffusion or drift of photogenerated carriers but also realize a self-powered operation. Here, a monolayer-multilayer graphene junction photodetector is accomplished by selectively thinning part of a multilayer graphene to a high-quality monolayer. Benefiting from the large photoabsorption cross section of multilayer graphene and strong asymmetry caused by the significant differences in optoelectronic properties between monolayer and multilayer graphene, the monolayer-multilayer graphene junction shows a 7-fold increase in short-circuit photocurrent as compared with that at the monolayer graphene-metal contact in scanning photocurrent images. The asymmetric configuration also enables the photodetector to work at zero bias with minimized dark current noise and stand-by power consumption. Under global illumination with visible light, a photoswitching ratio of 3.4 × 103, a responsivity of 8.8 mA W−1, a specific detectivity of 1.3 × 108 Jones and a response time of 11 ns can be obtained, suggesting a promising photoresponse. Moreover, it is worth mentioning that such a performance enhancement is achieved without compromising the broadband spectral response of graphene photodetector and it is hence applicable for long wavelength spectral range including infrared and terahertz.

19‐1: Invited Paper: Zonal Illuminated Non‐emissive Displays for AR Glass

LCoS (Liquid Crystal on Silicon) and DLP (Digital Light Processing) are mature non‐emissive display technologies. Both conventional LCoS and DLP use global illumination so that the whole display panel is illuminated regardless of the content. Therefore,it has challenges of high power consumption, low contrast ratio and so on. In this paper, we proposed a zonal illuminated LCoS and DLP systems, which greatly improve the contrast ratio and reduce the gray box artifacts. In addition, it saves >3x power consumption for typical sparse AR content.

Exploring the Multi-Sensory Coupling Relationship of Open Space on a Winter Campus

Exploring the combined effects of multisensory interactions in open spaces can help improve the comfort of campus environments. Nine typical spaces on a university campus in Fuzhou were selected for this study. Subjects perceived the environment and then completed an on-site subjective questionnaire. At the same time, meteorological data (global radiation, air temperature, globe temperature, wind speed, relative humidity, and illumination intensity) were measured to determine the interactions between visual and acoustic and thermal perceptions. Differences in the meteorological parameters between the measuring points were described using a one-way ANOVA and Tukey’s post hoc test, and a chi-square test of independence was used to determine significant associations between thermal, acoustic, and visual comfort, which in turn led to the study of interactions between visual, acoustic, and thermal comfort using a two-way ANOVA. The following conclusions were drawn: (1) the Thermal Comfort Vote (TCV) increased with the increasing Acoustic Comfort Vote (ACV) at all levels of thermal stress. (2) The highest and lowest Acoustic Sensation Vote (ASV) values for each sound type were derived from either “slightly cold” or “warm” conditions. Both the Thermal Comfort Vote (TCV) and the Acoustic Comfort Vote (ACV) were positively correlated. (3) When “neutral”, the Thermal Sensation Vote (TSV) increased with increasing illumination intensity (LUX). (4) The Sunlight Sensation Vote (SSV) increased with the increasing Universal Thermal Climate Index (UTCI) when illumination intensity (LUX) was moderate and bright. (5) The highest and lowest Acoustic Sensation Vote (ASV) values for each sound type came from either “slightly cold” or “warm” conditions.

A Fast GPU Schedule For À-Trous Wavelet-Based Denoisers

Given limitations of contemporary graphics hardware, real-time ray-traced global illumination is only estimated using a few samples per pixel. This consequently causes stochastic noise in the resulting frame sequences which requires wide filter support during denoising for temporally stable estimates. The edge avoiding à-trous wavelet transform amortizes runtime cost by hierarchical filtering using a constant number of increasingly dilated taps in each iteration. While the number of taps stays constant, the runtime of each iteration increases in these usually memory-throughput bound shaders with increasing dilation, because the increasing non-locality negatively impacts cache hit rates. We present a scheduling approach that optimizes usage of the memory subsystem by permutating global invocation indices in such a way that each wavelet filter iteration is applied through undilated taps. In contrast to prior approaches, our method has identical performance characteristics in each iteration, effectively decreasing maintenance cost and improving performance predictability. Furthermore, we are able to leverage on-chip memory and hardware texture interpolation. Our permutation strategy is trivial to integrate into existing wavelet filters as a permutation before and after each level of the wavelet filter. We achieve speedups between 1.3 and 3.8 for usual wavelet configurations in Monte Carlo denoising and computational photography.

Real‐time Neural Rendering of Dynamic Light Fields

AbstractSynthesising high‐quality views of dynamic scenes via path tracing is prohibitively expensive. Although caching offline‐quality global illumination in neural networks alleviates this issue, existing neural view synthesis methods are limited to mainly static scenes, have low inference performance or do not integrate well with existing rendering paradigms. We propose a novel neural method that is able to capture a dynamic light field, renders at real‐time frame rates at 1920× resolution and integrates seamlessly with Monte Carlo ray tracing frameworks. We demonstrate how a combination of spatial, temporal and a novel surface‐space encoding are each effective at capturing different kinds of spatio‐temporal signals. Together with a compact fully‐fused neural network and architectural improvements, we achieve a twenty‐fold increase in network inference speed compared to related methods at equal or better quality. Our approach is suitable for providing offline‐quality real‐time rendering in a variety of scenarios, such as free‐viewpoint video, interactive multi‐view rendering, or streaming rendering. Finally, our work can be integrated into other rendering paradigms, e.g., providing a dynamic background for interactive scenarios where the foreground is rendered with traditional methods.

Research on trajectory control of multi‐degree‐of‐freedom industrial robot based on visual image

AbstractIn order to improve the trajectory control effect of multi‐degree‐of‐freedom industrial robots, this paper combines visual image technology to conduct research on trajectory control of multi‐degree‐of‐freedom industrial robots. Aiming at the problem of video segmentation under sudden illumination changes, this paper uses a Gaussian mixture model based on the global illumination function to adopt a variety of illumination invariant features, and proposes a scene segmentation algorithm suitable for sudden illumination changes. Moreover, this paper compares and verifies the algorithm from the subjective and objective perspectives through experiments, which shows that the algorithm in this paper can segment the scene more accurately even in the environment of sudden changes in illumination. In addition, the results of the accuracy test and the trajectory control test show that the research method of the multi‐degree‐of‐freedom industrial robot trajectory control based on the visual image proposed in this paper can effectively improve the trajectory control effect of the robot.

Content-illumination coupling guided low-light image enhancement network

Current low-light enhancement algorithms fail to suppress noise when enhancing brightness, and may introduces structural distortion and color distortion caused by halos or artifacts. This paper proposes a content-illumination coupling guided low-light image enhancement network (CICGNet), it develops a truss topology based on Retinex as backbone to decompose low-light image component in an end-to-end way. The preservation of content features and the enhancement of illumination features are carried out along with depth and width direction of the truss topology. Each submodule uses the same resolution input and output to avoid the introduction of noise. Illumination component prevents misestimation of global and local illumination by using pre- and post-activation features at different depth levels, this way could avoid possible halos and artifacts. The network progressively enhances the illumination component and maintains the content component stage-by-stage. The proposed algorithm demonstrates better performance compared with advanced attention-based low-light enhancement algorithms and state-of-the-art image restoration algorithms. We also perform extensive ablation studies and demonstrate the impact of low-light enhancement algorithm on the downstream task of computer vision. Code is available at: https://github.com/Ruini94/CICGNet.

A real-time approximation algorithm of nonlinear filtering based on image pyramid

In the field of real-time rendering, post-processing filters are crucial techniques for achieving global illumination and photorealistic rendering in real-time rasterization and ray tracing rendering pipelines. Due to the power consumption limitations and insufficient memory bandwidth of mobile devices, filters can cause significant performance bottlenecks. To reduce the cost of filters on mobile devices, this paper proposes a real-time approximation method for nonlinear filtering. Our method reduce the number of texture samplings and memory usage. Through experiments conducted on mobile devices, it has been demonstrated that the proposed method can effectively reduce the number of texture samplings in feature images within scenes of similar complexity, as compared to mainstream industrial methods. This reduction in texture samplings leads to improved rendering quality while maintaining the same frame rate. Our method eliminates performance bottlenecks and enhances the quality of global illumination on mobile devices. By reducing the computational complexity of the non-linear filter, it provides users with a smoother experience.

Dark Light Image-Enhancement Method Based on Multiple Self-Encoding Prior Collaborative Constraints

The purpose of dark image enhancement is to restore dark images to visual images under normal lighting conditions. Due to the ill-posedness of the enhancement process, previous enhancement algorithms often have overexposure, underexposure, noise increases and artifacts when dealing with complex and changeable images, and the robustness is poor. This article proposes a new enhancement approach consisting in constructing a dim light enhancement network with more robustness and rich detail features through the collaborative constraint of multiple self-coding priors (CCMP). Specifically, our model consists of two prior modules and an enhancement module. The former learns the feature distribution of the dark light image under normal exposure as an a priori term of the enhancement process through multiple specific autoencoders, implicitly measures the enhancement quality and drives the network to approach the truth value. The latter fits the curve mapping of the enhancement process as a fidelity term to restore global illumination and local details. Through experiments, we concluded that the new method proposed in this article can achieve more excellent quantitative and qualitative results, improve detail contrast, reduce artifacts and noise, and is suitable for dark light enhancement in multiple scenes.

LEIFR-Net: light estimation for implicit face relight network.

Relighting facial images based on estimated lighting distribution and intensity from image backgrounds and environments can lead to more natural and convincing effects across diverse settings. In this paper, we introduce the Light Estimation for Implicit Face Relight Network (LEIFR-Net), which we believe to be a novel approach that significantly improves upon current methodologies. Initially, we present a method to estimate global illumination from a single image. We then detail our approach for structurally disentangled relighting of faces using pixel-aligned implicit functions. Furthermore, we elaborate on constructing a paired synthetic dataset, which includes environments, maps of lighting distribution, albedo and relighted faces, utilizing a process we refer to as stable diffusion. Our experimental results, evaluated against specific benchmarks, demonstrate the effectiveness of LEIFR-Net in achieving more harmonious alignment of highlights and shadows with environmental lighting, surpassing the performance of other contemporary methods in this domain.

Implementing Many-Lights Rendering with IES-Based Lights

In recent years, many research projects on real-time rendering have focused on the introduction of global illumination effects in order to improve the realism of a virtual scene. The main goal of these works is to find a compromise between the achievable quality of the resulting rendering and the intrinsic high computational cost of global illumination. An established approach is based on the use of Virtual Point Lights, i.e., “fictitious” light sources that are placed on surfaces in the scene. These lights simulate the contribution of light rays emitted by the light sources and bouncing on different objects. Techniques using Virtual Point Lights are often called Many-Lights techniques. In this paper, we propose an extension of a real-time Many-Lights rendering technique characterized by the integration of photometric data in the process of Virtual Point Lights distribution. We base the definition of light sources and the creation of Virtual Point Lights on the description provided in the IES standard format, created by the Illuminating Engineering Society (IES).

Performance of Hydride Vapor Phase Epitaxy‐Grown GaInP Photovoltaics with Double‐Sided Al‐Containing Passivation Layers Under Air Mass 1.5 Global Solar Irradiation and Low‐Intensity Indoor Light Irradiation

High‐performance III–V photovoltaic devices have the potential for various applications; however, their high production cost represents a challenge for market expansion. Hydride vapor phase epitaxy (HVPE) is a fabrication technology that can reduce the epitaxial growth cost of III–V compound semiconductors. This study demonstrates the performances of HVPE‐grown GaInP photovoltaic devices for solar and indoor photovoltaic applications. Herein, HVPE‐grown GaInP photovoltaic devices with AlInP frontside and AlGaInP backside passivation layers are fabricated for the first time. The developed GaInP photovoltaic device measured under air mass 1.5 global solar spectrum illumination achieves an independently certified power conversion efficiency of 17.3%, which is a new record efficiency among HVPE‐grown GaInP single‐junction solar cells. Furthermore, the GaInP photovoltaic device grown using HVPE exhibits a high power conversion efficiency (34.7–37.1%) under an illumination of 500–2000 lux, respectively, emitted by a white light‐emitting diode. These results demonstrate that HVPE‐grown GaInP photovoltaic devices can be used in low‐intensity indoor light energy harvesting systems.

Decorrelating ReSTIR Samplers via MCMC Mutations

Monte Carlo rendering algorithms often utilize correlations between pixels to improve efficiency and enhance image quality. For real-time applications in particular, repeated reservoir resampling offers a powerful framework to reuse samples both spatially in an image and temporally across multiple frames. While such techniques achieve equal-error up to 100× faster for real-time direct lighting [Bitterli et al. 2020 ] and global illumination [Ouyang et al. 2021 ; Lin et al. 2021 ], they are still far from optimal. For instance, spatiotemporal resampling often introduces noticeable correlation artifacts, while reservoirs holding more than one sample suffer from impoverishment in the form of duplicate samples. We demonstrate how interleaving Markov Chain Monte Carlo (MCMC) mutations with reservoir resampling helps alleviate these issues, especially in scenes with glossy materials and difficult-to-sample lighting. Moreover, our approach does not introduce any bias, and in practice, we find considerable improvement in image quality with just a single mutation per reservoir sample in each frame.

Photon Field Networks for Dynamic Real-Time Volumetric Global Illumination.

Volume data is commonly found in many scientific disciplines, like medicine, physics, and biology. Experts rely on robust scientific visualization techniques to extract valuable insights from the data. Recent years have shown path tracing to be the preferred approach for volumetric rendering, given its high levels of realism. However, real-time volumetric path tracing often suffers from stochastic noise and long convergence times, limiting interactive exploration. In this paper, we present a novel method to enable real-time global illumination for volume data visualization. We develop Photon Field Networks-a phase-function-aware, multi-light neural representation of indirect volumetric global illumination. The fields are trained on multi-phase photon caches that we compute a priori. Training can be done within seconds, after which the fields can be used in various rendering tasks. To showcase their potential, we develop a custom neural path tracer, with which our photon fields achieve interactive framerates even on large datasets. We conduct in-depth evaluations of the method's performance, including visual quality, stochastic noise, inference and rendering speeds, and accuracy regarding illumination and phase function awareness. Results are compared to ray marching, path tracing and photon mapping. Our findings show that Photon Field Networks can faithfully represent indirect global illumination within the boundaries of the trained phase spectrum while exhibiting less stochastic noise and rendering at a significantly faster rate than traditional methods.

Generalizable Polyp Segmentation via Randomized Global Illumination Augmentation.

Accurately segmenting polyps from colonoscopy images is essential for diagnosing colorectal cancer. Despite the tremendous success of the deep convolutional neural networks in automatic polyp segmentation, it suffers from domain shift issues, where the trained model yields performance deterioration on unseen test datasets. This paper proposes an illumination enhancement-based domain generalization approach to improve the generalization capability of the model on unseen test datasets and alleviate this issue. In particular, an image decomposition module (IDM) was developed to separate colonoscopy images into reflectance, local, and global illumination components. An illumination transform module (ITM) was proposed to augment images with different global illuminations by synthesizing target-like global illumination maps. A novel illumination variance insensitiveness (IViSen) is also introduced to evaluate the robustness of the model against illumination disturbance. IViSen is easy to compute and correlates well with model generalizability. The segmentation performance of the proposed model on four colonoscopy datasets was examined: CVC-ClinicDB, CVC-ColonDB, ETIS-Larib, and Kvasir-SEG. The method outperformed the competitive methods when tested on unseen domains. In particular, the proposed approach yielded 60.82% and 53.19% in terms of mean Dice and IoU, respectively, with 2.06% and 2.31% improvements.

Efficient Binocular Rendering of Volumetric Density Fields With Coupled Adaptive Cube-Map Ray Marching for Virtual Reality.

Creating visualizations of multiple volumetric density fields is demanding in virtual reality (VR) applications, which often include divergent volumetric density distributions mixed with geometric models and physics-based simulations. Real-time rendering of such complex environments poses significant challenges for rendering quality and performance. This article presents a novel scheme for efficient real-time rendering of varying translucent volumetric density fields with global illumination (GI) effects on high-resolution binocular VR displays. Our scheme proposes creative solutions to address three challenges involved in the target problem. First, to tackle the doubled heavy workloads of binocular ray marching, we explore the anti-aliasing principles and more advanced potentials of ray marching on interior cube-map faces, and propose a coupled ray-marching technique that converges to multi-resolution cube maps with interleaved adaptive sampling. Second, we devise a fully dynamic ambient GI approximation method that leverages spherical-harmonics (SH) transform information of the phase function to reduce the huge amount of ray sampling required for GI while ensuring fidelity. The method catalyzes spatial ray-marching reuse and adaptive temporal accumulation. Third, we deploy a two-phase ray-tracing algorithm with a tiled k-buffer to achieve fast processing of order-independent transparency (OIT) for multiple volume instances. Consequently, high-quality and high-performance real-time dynamic volume rendering can be achieved under constrained budgets controlled by developers. As our solution supports mixed mesh-volume rendering, the test results prove the practical usefulness of our approach for high-resolution binocular VR rendering on hybrid multi-volumetric and geometric environments.

The expression of multimedia animation art design in the era of artificial intelligence

Abstract Aesthetic rules are the basis of animation art design, combined with the user-centered design method is a new form of multimedia animation art design. This paper combines the existing three-dimensional animation art forms of expression, the use of points, lines, and surfaces to generate three-dimensional animation three-dimensional space, and selected virtual reality technology to achieve three-dimensional animation conversion. Based on ray tracing algorithm and radiance global illumination technology, to optimize animation scene realism, analyze the difficulties of rendering image styles. Combined with the Canny edge extraction algorithm to extract the target area contour, homogenization processing of local area coloring, fusion of the extracted contour and coloring area, the formation of animation style rendering, constituting the design of this paper three-dimensional animation art design expression. Analyze the rendering data of trees in a typical natural scene and use design evaluation indexes to assess the performance impact of animation scene art design. The average score for the animation scene expressing information content, the average score for the animation matching with the information, and the average score for the animation effect art and beauty are 7.672, 7.027, and 7.474, respectively. The scores indicate that the animation art design expression proposed in this paper can provide a more accurate depiction of the aesthetics of animation art.

RGB-T Tracking with Template-Bridged Search Interaction and Target-Preserved Template Updating.

The goal of RGB-Thermal (RGB-T) tracking is to utilize the synergistic and complementary strengths of RGB and TIR modalities to enhance tracking in diverse situations, with cross-modal interaction being a crucial element. Earlier methods often simply combine the features of the RGB and TIR search frames, leading to a coarse interaction that also introduced unnecessary background noise. Many other approaches sample candidate boxes from search frames and apply different fusion techniques to individual pairs of RGB and TIR boxes, which confines cross-modal interactions to local areas and results in insufficient context modeling. Additionally, mining video temporal contexts is also under-explored in RGB-T tracking. To alleviate these limitations, we propose a novel Template-Bridged Search region Interaction (TBSI) module that exploits templates as the medium to bridge the cross-modal interaction between RGB and TIR search regions by gathering and distributing target-relevant object and environment contexts. An Illumination Guided Fusion (IGF) module is designed to adaptively fuse RGB and TIR search region tokens with a global illumination factor. Furthermore, in the inference stage, we also propose an efficient Target-Preserved Template Updating (TPTU) strategy, leveraging the temporal context within video sequences to accommodate the target's appearance change. Our proposed modules are integrated into a ViT backbone for joint feature extraction, search-template matching, and cross-modal interaction. Extensive experiments on three popular RGB-T tracking benchmarks demonstrate our method achieves new state-of-the-art performances. Code is available at https://github.com/RyanHTR/TBSI.