Ambient Occlusion Research Articles

DHR+S: distributed hybrid rendering with realistic real-time shadows for interactive thin client metaverse and game applications

Distributed hybrid rendering (DHR) is a real-time rendering approach that incorporates cloud-based ray tracing with locally rasterized graphics for interactive thin client metaverse and game applications. With cloud assistance, DHR can generate high-fidelity ray-traced graphics contents remotely and deliver them to thin clients with low graphics capability, including standalone extended reality devices and mobile phones, while maintaining interactive frame rates for users under adverse network conditions. DHR can already achieve the effect of ray-traced hard shadows that form with the occlusion of direct illumination. We enhance the realism of these shadows by softening their edges with the direction of rays traced and approximating the occlusion of indirect illumination by reconstructing ray-traced ambient occlusion with a modified version of spatiotemporal variance-guided filtering. Our technique uses only 20–30% of the bandwidth of remote rendering and is also tolerant of delays of up to 200 ms with only slight distortion to the shadows along object edges.

Ambient Occlusion for Terrain Shading

Ambient occlusion is widely used in computer graphics to accentuate areas in a three-dimensional scene that are hidden from bright lighting. On maps, ambient occlusion adds spatial depth and nuanced texture to shaded relief images. Ambient occlusion images for maps can be calculated from digital elevation models using the Eduard relief shading application. This paper illustrates the use of ambient occlusion for cartographic terrain shading and introduces two extensions: (1) oriented ambient occlusion for modulation with a direction of illumination, and (2) reduced ambient occlusion for valley bottoms with consistent brightness.

FAST: Filter-Adapted Spatio-Temporal Sampling for Real-Time Rendering

Stochastic sampling techniques are ubiquitous in real-time rendering, where performance constraints force the use of low sample counts, leading to noisy intermediate results. To remove this noise, the post-processing step of temporal and spatial denoising is an integral part of the real-time graphics pipeline. The main insight presented in this paper is that we can optimize the samples used in stochastic sampling such that the post-processing error is minimized. The core of our method is an analytical loss function which measures post-filtering error for a class of integrands --- multidimensional Heaviside functions. These integrands are an approximation of the discontinuous functions commonly found in rendering. Our analysis applies to arbitrary spatial and spatiotemporal filters, scalar and vector sample values, and uniform and non-uniform probability distributions. We show that the spectrum of Monte Carlo noise resulting from our sampling method is adapted to the shape of the filter, resulting in less noisy final images. We demonstrate improvements over state-of-the-art sampling methods in three representative rendering tasks: ambient occlusion, volumetric ray-marching, and color image dithering. Common use noise textures, and noise generation code is available at https://github.com/electronicarts/fastnoise.

The Algorithm for Generating Ambient Occlusion Maps Based on the Differential Algorithm

This paper introduces an algorithm for generating ambient occlusion maps based on the differential algorithm, aiming to enhance the environmental occlusion effects in graphic rendering through advanced computational techniques. The algorithm is inspired by the principles of the differential algorithm and the lighting information stored in normal maps. By inputting a differential step size, it cleverly samples and computes the gradient information of the normal map, resulting in the generation of high-quality ambient occlusion maps. The unique advantage of the algorithm lies in its sensitivity to lighting information and its flexible application of the differential step size. By effectively capturing subtle gradient variations in the normal map, it provides a more realistic and refined effect for the final physically based rendering (PBR) material graphics rendering.

A smart fruit size measuring method and system in natural environment

The measurement of fruit size (length, width, and area) is one of the key components in the assessment of fruit ripeness for consumption and fruit quality. Due to issues with fruit shape and ambient occlusion, noncontact measurement of fruits in their natural growth state in natural environment is more difficult. This study proposes a fast and low-cost noncontact measurement solution combining deep learning, image analysis and robotic platform. First, based on the YOLOv5 object detection algorithm and converged dataset, a 95.6% detection accuracy and 0.05 s detection speed was achieved. Second, a Cycle GAN model for fruit occlusion recovery was established, and the occlusion of the fruit under different natural conditions was adaptively repaired, with a 5.48% average relative error under different occlusion conditions. Third, through image morphological operations, the size measurements were achieved, with an average relative error of 8.43% for individual fruits and the 10.12% overall error under the complex natural environment. This method and platform provide a systematic, fast (0.2s) and low-cost solution for the noncontact measurement of fruit size.

Contextual Ambient Occlusion: A volumetric rendering technique that supports real-time clipping

In this paper, we present a new volumetric ambient occlusion algorithm called Contextual Ambient Occlusion (CAO) that supports real-time clipping. The algorithm produces ambient occlusion images of exactly the same quality as Local Ambient Occlusion (LAO) while enabling real-time modification to the shape used to clip the volume. The main idea of the algorithm is that clipping only affects the ambient value of a small number of voxels, so by identifying these voxels and recalculating the ambient factor only for them, it is possible to significantly increase the rendering performance (by 2-5x) without decreasing the quality of the rendered image. Due to its fast performance, the algorithm is suitable for interactive environments where clipping changes could occur every frame. Additionally, the algorithm does not have any stereoscopic inconsistency, which makes it suitable for mixed reality environments. This paper is an extended version of the “Contextual Ambient Occlusion” article presented during the 2023 Graphics Interface conference, and includes, among other additions, the source code of the algorithm.

CoordNet: Data Generation and Visualization Generation for Time-Varying Volumes via a Coordinate-Based Neural Network.

Although deep learning has demonstrated its capability in solving diverse scientific visualization problems, it still lacks generalization power across different tasks. To address this challenge, we propose CoordNet, a single coordinate-based framework that tackles various tasks relevant to time-varying volumetric data visualization without modifying the network architecture. The core idea of our approach is to decompose diverse task inputs and outputs into a unified representation (i.e., coordinates and values) and learn a function from coordinates to their corresponding values. We achieve this goal using a residual block-based implicit neural representation architecture with periodic activation functions. We evaluate CoordNet on data generation (i.e., temporal super-resolution and spatial super-resolution) and visualization generation (i.e., view synthesis and ambient occlusion prediction) tasks using time-varying volumetric data sets of various characteristics. The experimental results indicate that CoordNet achieves better quantitative and qualitative results than the state-of-the-art approaches across all the evaluated tasks. Source code and pre-trained models are available at https://github.com/stevenhan1991/CoordNet.

The role of different light settings on the perception of realism in virtual replicas in immersive Virtual Reality

Immersive Virtual Reality (IVR) provides a platform where the real world can be replicated to a point where users can act and react in the virtual world as they would in reality. However, rendering visual stimuli is computationally heavy. Thus, optimizations must be done to take advantage of computational systems by studying our perception of reality. This study investigated parameters related to light rendering (Global Illumination, Ambient Occlusion, Screen Space Reflections (SSR) and Direct Shadows) in real-time in a virtual replica of a real place using IVR. Participants experienced both virtual and real rooms with only one flashlight and changed the quality settings of the considered parameters so that their sense of reality would be the closest to the one they felt when they experienced the real room. Participants were given a budget to drive them to prioritize what parameters, and their level of quality, are the most important for their sense of reality. Results indicated that participants considered Global Illumination the most important factor, closely followed by Direct Shadows. Ambient Occlusion and Reflections (Screen Space Reflections) were the less prioritized parameters. We conclude that in a lighting setting where only dynamic lights are used, Global Illumination and Direct Shadows should be prioritized over SSR Reflections and Ambient Occlusion when computational power is limited.

Jrender: An efficient differentiable rendering library based on Jittor

Differentiable rendering has been proven as a powerful tool to bridge 2D images and 3D models. With the aid of differentiable rendering, tasks in computer vision and computer graphics could be solved more elegantly and accurately. To address challenges in the implementations of differentiable rendering methods, we present an efficient and modular differentiable rendering library named Jrender based on Jittor. Jrender supports surface rendering for 3D meshes and volume rendering for 3D volumes. Compared with previous differentiable renderers, Jrender exhibits a significant improvement in both performance and rendering quality. Due to the modular design, various rendering effects such as PBR materials shading, ambient occlusions, soft shadows, global illumination, and subsurface scattering could be easily supported in Jrender, which are not available in other differentiable rendering libraries. To validate our library, we integrate Jrender into applications such as 3D object reconstruction and NeRF, which show that our implementations could achieve the same quality with higher performance.

Finding Nano-Ötzi: Cryo-Electron Tomography Visualization Guided by Learned Segmentation.

Cryo-electron tomography (cryo-ET) is a new 3D imaging technique with unprecedented potential for resolving submicron structural details. Existing volume visualization methods, however, are not able to reveal details of interest due to low signal-to-noise ratio. In order to design more powerful transfer functions, we propose leveraging soft segmentation as an explicit component of visualization for noisy volumes. Our technical realization is based on semi-supervised learning, where we combine the advantages of two segmentation algorithms. First, the weak segmentation algorithm provides good results for propagating sparse user-provided labels to other voxels in the same volume and is used to generate dense pseudo-labels. Second, the powerful deep-learning-based segmentation algorithm learns from these pseudo-labels to generalize the segmentation to other unseen volumes, a task that the weak segmentation algorithm fails at completely. The proposed volume visualization uses deep-learning-based segmentation as a component for segmentation-aware transfer function design. Appropriate ramp parameters can be suggested automatically through frequency distribution analysis. Furthermore, our visualization uses gradient-free ambient occlusion shading to further suppress the visual presence of noise, and to give structural detail the desired prominence. The cryo-ET data studied in our technical experiments are based on the highest-quality tilted series of intact SARS-CoV-2 virions. Our technique shows the high impact in target sciences for visual data analysis of very noisy volumes that cannot be visualized with existing techniques.

RadarSense: Accurate Recognition of Mid-air Hand Gestures with Radar Sensing and Few Training Examples

Microwave radars bring many benefits to mid-air gesture sensing due to their large field of view and independence from environmental conditions, such as ambient light and occlusion. However, radar signals are highly dimensional and usually require complex deep learning approaches. To understand this landscape, we report results from a systematic literature review of ( N =118) scientific papers on radar sensing, unveiling a large variety of radar technology of different operating frequencies and bandwidths and antenna configurations but also various gesture recognition techniques. Although highly accurate, these techniques require a large amount of training data that depend on the type of radar. Therefore, the training results cannot be easily transferred to other radars. To address this aspect, we introduce a new gesture recognition pipeline that implements advanced full-wave electromagnetic modeling and inversion to retrieve physical characteristics of gestures that are radar independent, i.e., independent of the source, antennas, and radar-hand interactions. Inversion of radar signals further reduces the size of the dataset by several orders of magnitude, while preserving the essential information. This approach is compatible with conventional gesture recognizers, such as those based on template matching, which only need a few training examples to deliver high recognition accuracy rates. To evaluate our gesture recognition pipeline, we conducted user-dependent and user-independent evaluations on a dataset of 16 gesture types collected with the Walabot, a low-cost off-the-shelf array radar. We contrast these results with those obtained for the same gesture types collected with an ultra-wideband radar made of a vector network analyzer with a single horn antenna and with a computer vision sensor, respectively. Based on our findings, we suggest some design implications to support future development in radar-based gesture recognition.

Effect-based Multi-viewer Caching for Cloud-native Rendering

With cloud computing becoming ubiquitous, it appears as virtually everything can be offered as-a-service. However, real-time rendering in the cloud forms a notable exception, where the cloud adoption stops at running individual game instances in compute centers. In this paper, we explore whether a cloud-native rendering architecture is viable and scales to multi-client rendering scenarios. To this end, we propose world-space and on-surface caches to share rendering computations among viewers placed in the same virtual world. We discuss how caches can be utilized on an effect-basis and demonstrate that a large amount of computations can be saved as the number of viewers in a scene increases. Caches can easily be set up for various effects, including ambient occlusion, direct illumination, and diffuse global illumination. Our results underline that the image quality using cached rendering is on par with screen-space rendering and due to its simplicity and inherent coherence, cached rendering may even have advantages in single viewer setups. Analyzing the runtime and communication costs, we show that cached rendering is already viable in multi-GPU systems. Building on top of our research, cloud-native rendering may be just around the corner.

Quantification of Active Visual Attention using RGB camera.

Active visual attention (AVA) is the cognitive ability that helps to focus on important visual information while responding to a stimulus and is important for human-behavior and psychophysiological research. Existing eye-trackers/camera-based methods are either expensive or impose privacy issues as face videos are recorded for analysis. Proposed approach using blink-rate variability (BRV), is inexpensive, easy to implement, efficient and handles privacy issues, making it amenable to real-time applications. Our solution uses laptop camera/webcams and a single blink feature, namely BRV. First, we estimated participant's head pose to check camera alignment and detect if he is looking at the screen. Next, subject-specific threshold is computed using eye aspect ratio (EAR) to detect blinks from which BRV signal is constructed. Only EAR values are saved, and participant's face video is NOT saved or transmitted. Finally, a novel AVA score is computed. Results shows that the proposed score is robust across participants, ambient light conditions and occlusions like spectacles.

HISTORIC STONE PAVEMENT PHOTOGRAMMETRIC 3D-SURVEY: A WAY TO GET A CATALOGUE TO PASS ON ANCIENT CRAFTS

Abstract. In Tuscany, the art of paving has never been codified in written form, being already encapsulated in knowledge based on practical experience, with specialized tradesmen and stonemasons relying on direct observation and passing on their knowledge to apprentices through repetition of gestures. The streets of Volterra, a historic town in Tuscany, are still paved with the local stone, the panchina volterrana (Volterran panchina stone). Photogrammetry techniques allow high-resolution reconstruction of 3-D pavement models and geometric representation of the macrotexture component resulting from the chiselling of the stone, as well as gaining high-resolution, high-precision geometry information along with colour information. In this way, it is possible to fully characterize the stone and generate a catalogue, which can become an operational tool for those involved in monitoring and maintaining pavements. This paper presents the methodology of photogrammetric surveying and the possibility of obtaining indices of the macrotexture of the processed surface of the stone, in terms of Virtual Mean Texture Depth (V-MTD), in analogy with the analysis of asphalt surfaces. The classification is also proposed by integrating the geometric macrotexture data with the data derived from a segmentation of the different textures obtainable for the model (diffuse, ambient occlusion). This geometric information could be then enriched by the stonemason's indications, constituting, therefore, a tool for passing on one's knowledge and experience.

Симулятор виртуальной реальности по оказанию первой медицинской помощи для использования при обучении персонала АСУП

The article considers the proposed models and methods of designing, developing and implementing a virtual reality VR simulator on first aid for training specialists of the staff management subsystem of the automated pro-duction management system. Analysis of information about the injured at work in the territory of the Russian Federation is given, according to the data of the Federal State Statistics Service. The system structure has been developed. There is proposed a new approach to creating more realistic 3D models for visualizing in virtual reality, developing clothes for 3D models of people in virtual reality in the Marvelous designer software, and a strategy for optimizing 3D models for loading them into a virtual reality application. The process of model development, types of incorrect and correct model topology, types of high-polygonal model before retopology and low-polygonal model after retopology, the process of baking a normal map in Substance Painter software and a baked normal map, a model without and with a normal map, light adjustment, texturing and baking maps, materials consisting of PBR textures and main maps Albedo, Metallic, Roughness, Normal, Ambient Occlusion, Emission maps and models with and without textures, shaders. There is considered an approach to creating clothing for a human model in ZBrush software, as well as a seam on clothing for 3D models in virtual reality. Algorithms for real-time accounting, interaction with three-dimensional models in real time, as well as an algorithm for the operation of animated objects in a virtual reality simulator have been developed. The simulator has been tested in training the staff of the Moscow state budgetary institution the Center for Innovative Technologies in Leisure and Sports “Progress” and in the professional training program for the profession Car Driver implemented by SBVEI 26KADR in 2019. The positive results of the implementation are described.

The Representational Similarity between Visual Perception and Recent Perceptual History.

From moment to moment, the visual properties of objects in the world fluctuate because of external factors like ambient lighting, occlusion and eye movements, and internal (proximal) noise. Despite this variability in the incoming information, our perception is stable. Serial dependence, the behavioral attraction of current perceptual responses toward previously seen stimuli, may reveal a mechanism underlying stability: a spatiotemporally tuned operator that smooths over spurious fluctuations. The current study examined the neural underpinnings of serial dependence by recording the electroencephalographic (EEG) brain response of female and male human observers to prototypical objects (faces, cars, and houses) and morphs that mixed properties of two prototypes. Behavior was biased toward previously seen objects. Representational similarity analysis (RSA) revealed that responses evoked by visual objects contained information about the previous stimulus. The trace of previous representations in the response to the current object occurred immediately on object appearance, suggesting that serial dependence arises from a brain state or set that precedes processing of new input. However, the brain response to current visual objects was not representationally similar to the trace they leave on subsequent object representations. These results reveal that while past stimulus history influences current representations, this influence does not imply a shared neural code between the previous trial (memory) and the current trial (perception).SIGNIFICANCE STATEMENT The perception of visual objects is pulled toward instances of that object seen in the recent past. The neural underpinnings of this serial dependence remain to be fully investigated. The present study examined electroencephalographic (EEG) responses to faces, cars, and houses, and ambiguous between-category morphs. With representational similarity analysis (RSA), we showed (1) object-specific neural patterns that differentiate the three categories; (2) that the response to the current object contains information about the previous object, mirroring behavioral serial dependence; (3) that the object-specific neural pattern about the past was different from that in the current response, revealing that while past stimulus history influences current representations, this does not imply a shared neural code between the previous trial (memory) and the current trial (perception).

Luminance contrast provides metric depth information.

The perception of depth from retinal images depends on information from multiple visual cues. One potential depth cue is the statistical relationship between luminance and distance; darker points in a local region of an image tend to be farther away than brighter points. We establish that this statistical relationship acts as a quantitative cue to depth. We show that luminance variations affect depth in naturalistic scenes containing multiple cues to depth. This occurred when the correlation between variations of luminance and depth was manipulated within an object, but not between objects. This is consistent with the local nature of the statistical relationship in natural scenes. We also showed that perceived depth increases as contrast is increased, but only when the depth signalled by luminance and binocular disparity are consistent. Our results show that the negative correlation between luminance and distance, as found under diffuse lighting, provides a depth cue that is combined with depth from binocular disparity, in a way that is consistent with the simultaneous estimation of surface depth and reflectance variations. Adopting more complex lighting models such as ambient occlusion in computer rendering will thus contribute to the accuracy as well as the aesthetic appearance of three-dimensional graphics.

Multi feature-rich synthetic colour to improve human visual perception of point clouds

Although point features have shown their usefulness in classification with Machine Learning, point cloud visualization enhancement methods focus mainly on lighting. The visualization of point features helps to improve the perception of the 3D environment. This paper proposes Multi Feature-Rich Synthetic Colour (MFRSC) as an alternative non-photorealistic colour approach of natural-coloured point clouds. The method is based on the selection of nine features (reflectance, return number, inclination, depth, height, point density, linearity, planarity, and scattering) associated with five human perception descriptors (edges, texture, shape, size, depth, orientation). The features are reduced to fit the RGB display channels. All feature permutations are analysed according to colour distance with the natural-coloured point cloud and Image Quality Assessment. As a result, the selected feature permutations allow a clear visualization of the scene's rendering objects, highlighting edges, planes, and volumetric objects. MFRSC effectively replaces natural colour, even with less distorted visualization according to BRISQUE, NIQUE and PIQE. In addition, the assignment of features in RGB channels enables the use of MFRSC in software that does not support colorization based on point attributes (most commercially available software). MFRSC can be combined with other non-photorealistic techniques such as Eye-Dome Lighting or Ambient Occlusion.

On-Tube Attribute Visualization for Multivariate Trajectory Data.

Stylized tubes are an established visualization primitive for line data as encountered in many scientific fields, ranging from characteristic lines in flow fields, fiber tracks reconstructed from diffusion tensor imaging, to trajectories of moving objects as they arise from cyber-physical systems in many engineering disciplines. Typical challenges include large data set sizes demanding for efficient rendering techniques as well as a large number of attributes that cannot be mapped simultaneously to the basic visual attributes provided by a tube-based visualization. In this work, we tackle both challenges with a new on-tube visualization approach. We improve recent work on high-quality GPU ray casting of Hermite spline tubes supporting ambient occlusion and extend it by a new layered procedural texturing technique. In the proposed framework, a large number of data set attributes can be mapped simultaneously to a variety of glyphs and plots that are embedded in texture space and organized in layers. Efficient rendering with minimal data transfer is achieved by generating the glyphs procedurally and drawing them in a deferred shading pass. We integrated these techniques in a prototype visualization tool that facilitates flexible mapping of data set attributes to visual tube and glyph attributes. We studied our approach on a variety of example data from different fields and found it to provide a highly adaptable and extensible toolbox to quickly craft tailor-made tube-based trajectory visualizations.

Light weight convolutional neural network and low-dimensional images transformation approach for classification of thermal images

Thermal energy is emitted in the infrared range between X-ray and Gamma rays, which are invisible to the human eye. Thermal cameras can detect the temperature that arises due to the heat emitted by the objects in a non-contact way and transform it into an image. These images ensure to detection of objects regardless of ambient occlusion. Based on this problem, five different classification models were proposed within the scope of the study. New low-dimensional images were obtained by extracting the features of thermal images with HOG (Histogram Oriented of Gradients), LBP (Local Binary Pattern), SIFT (Scale Invariant Feature Transform), and GF (Gabor Filter) methods. These images are classified by a CNN (Convolutional Neural Network) model called LW-CNN (Light Weight CNN). Raw thermal images were classified with the LW-CNN model without pre-processing. In order to analyze the efficiency of the proposed models, the results were compared via the pre-trained VGG16 model. Three different datasets containing thermal images were used in classification processes. The highest classification accuracy was obtained from the LW-CNN model in the performance evaluations carried out on the three datasets. With this model, the classification accuracies obtained from the datasets are 98.58%, 95.56%, and 100%, respectively.