Volume Encoding Gaussians: Transfer Function-Agnostic 3D Gaussians for Volume Rendering.

Introduction

The graphical representation of quantitative information is not a modern development, but rather it can be traced back to the earliest map-making and, later, thematic cartography and statistical graphics (Friendly, 2008). The early 19th century witnessed the invention of all major forms of statistical graphics, including the ever so popular pie and bar charts, histograms, line graphs, and scatterplots. At this time, data from a wide variety of domains (e.g., economic, social, medical, physical) began to be depicted, and a wide range of novel techniques were used to facilitate data representation. At the same time, graphical analyses of natural and physical phenomena made regular appearances in scientific publications. In the second half of the 19th century, there was a rapid growth in the visualization of data: the importance of numerical information for public policy, industry, and health was acknowledged, and the various applications of statistical theory and methods made it easier to make sense of large bodies of data. This period has been referred to “the Golden Age” of data visualization (Friendly, 2008, pp. 12–13). Another historically critical period of the development of data visualization is between 1950 and 1975 (Friendly, 2008). In this period, data analysis began being recognized as a distinct branch of statistics by the international research community and significant advances were made in the area of computer processing of statistical data, interactive statistical applications, and digital graphic technologies. Since the mid-1970s, data visualization has blossomed into a vibrant multi-disciplinary research area. It features characteristics such as highly interactive statistical computing systems, advanced visualizations of high-dimensional data, and substantially increased attention to the cognitive and perceptual aspects of data display. Data representations and visualizations have also become commonplace in the applied work of a variety of professions. Scientists, for example, use data visualizations to make sense of trends within their research that employs mathematical and statistical models of phenomena and make such results understandable by others. Engineers use data representations to monitor environmental, commercial, and industrial processes. Historians and journalists also utilize data representations and visualizations to communicate information from a myriad of sources, including textual data. Finally, more recently, individuals – even students – have begun to use data representations and visualizations to understand aspects of their lives, such as their wellness and finances (Lee, Choe, Isenberg, Marriott, & Stasko, 2020). Not left behind the data revolution are educational researchers, who use data representations and visualizations, which we use synonymously in this article, for many of the same reasons as other professionals and non-professionals – to understand and communicate results effectively. While many engage with data representations and visualizations, a focus on the effectiveness of their design has often been ignored (Wilkinson, 2005). However, after a period where data representations and visualizations were seen by statisticians as “a minor subfield and are not well-integrated with larger themes of modeling and inference” (Gelman & Unwin, 2013, p. 1), many professionals are beginning to take representation and visualization seriously. This is evidenced by the recent theoretical and practical work that is being done by the likes of Healy (2018), Wickham (2016), and Wilkinson (2005). Moreover, there is research and recent work in the broader fields of computer science, statistics, and sociology, to name a few, that can inform how we, as educational researchers, go about creating data representations and visualizations effectively. Finally, as we begin considering effective ways to represent and visualize data, it is important to consider findings from educational, psychological, and developmental research on how people interpret data representations and visualizations as we make related decisions. Thus, there is, presently, a greater focus on the effectiveness of data representations and visualizations, a focus which we aim to highlight and demonstrate through this article.

To accurately represent object position in real time, the human visual system predictively encodes the location of moving objects, compensating for the time required for transmission and processing of information.

Direct volume rendering techniques allow visualization of volume data without extracting intermediate geometry. The mapping from voxel attributes to optical properties is performed by transfer functions which, consequently, play a crucial role in building informative images from the data. One-dimensional transfer functions, which are based only on a scalar value per voxel, often do not provide proper visualizations. On the other hand, multidimensional transfer functions can perform more sophisticated data classification, based on vectorial voxel signatures. The transfer function design is a non-trivial and unintuitive task, especially in the multi-dimensional case. In this paper we propose a multi-dimensional transfer function design technique that uses self-organizing maps to perform dimensional reduction. Our approach gives uniform treatment to volume data containing voxel signatures of arbitrary dimension, and allows the use of any type of voxel attribute as part of the voxel signatures.

Direct volume rendering is an important tool for visualizing complex data sets. However, in the process of generating 2D images from 3D data, information is lost in the form of attenuation and occlusion. The lack of a feedback mechanism to quantify the loss of information in the rendering process makes the design of good transfer functions a difficult and time consuming task. In this paper, we present the notion of visibility-driven transfer functions, which are transfer functions that provide a good visibility of features of interest from a given viewpoint. To achieve this, we introduce visibility histograms. These histograms provide graphical cues that intuitively inform the user about the contribution of particular scalar values to the final image. By carefully manipulating the parameters of the opacity transfer function, users can now maximize the visibility of the intervals of interest in a volume data set. Based on this observation, we also propose a semi-automated method for generating transfer functions, which progressively improves a transfer function defined by the user, according to a certain importance metric. Now the user does not have to deal with the tedious task of making small changes to the transfer function parameters, but now he/she can rely on the system to perform these searches automatically. Our methodology can be easily deployed in most visualization systems and can be used together with traditional 1D opacity transfer functions based on scalar values, as well as with multidimensional transfer functions and other more sophisticated rendering algorithms.

This paper presents tools recently developed at CEA/Gramat to analyse the transient electromagnetic field on large targets. It focusses on: • The work on GORF3D FDTD solver and its peripheral tools developed in order to study very large and complex scenes • The results of numerical study of external and internal coupling from a plane wave to a very large scene • The difference on external and internal coupling results between a building taken alone or placed in a very large scene.

We examine visual representations of data that make use of combinations of both 2D and 3D data mappings. Combining 2D and 3D representations is a common technique that allows viewers to understand multiple facets of the data with which they are interacting. While 3D representations focus on the spatial character of the data or the dedicated 3D data mapping, 2D representations often show abstract data properties and take advantage of the unique benefits of mapping to a plane. Many systems have used unique combinations of both types of data mappings effectively. Yet there are no systematic reviews of the methods in linking 2D and 3D representations. We systematically survey the relationships between 2D and 3D visual representations in major visualization publications-IEEE VIS, IEEE TVCG, and EuroVis-from 2012 to 2022. We closely examined 105 articles where 2D and 3D representations are connected visually, interactively, or through animation. These approaches are designed based on their visual environment, the relationships between their visual representations, and their possible layouts. Through our analysis, we introduce a design space as well as provide design guidelines for effectively linking 2D and 3D visual representations.

Transfer functions facilitate the volumetric data visualization by assigning optical properties to various data features and scalar values. Automation of transfer function specifications still remains a challenge in volume rendering. This paper presents an approach for automating transfer function generations by utilizing topological attributes derived from the contour tree of a volume. The contour tree acts as a visual index to volume segments, and captures associated topological attributes involved in volumetric data. A residue flow model based on Darcy's Law is employed to control distributions of opacity between branches of the contour tree. Topological attributes are also used to control color selection in a perceptual color space and create harmonic color transfer functions. The generated transfer functions can depict inclusion relationship between structures and maximize opacity and color differences between them. The proposed approach allows efficient automation of transfer function generations, and exploration on the data to be carried out based on controlling of opacity residue flow rate instead of complex low-level transfer function parameter adjustments. Experiments on various data sets demonstrate the practical use of our approach in transfer function generations.

Direct volume rendering is an important tool for visualizing complex data sets. However, in the process of generating 2D images from 3D data, information is lost in the form of attenuation and occlusion. The lack of a feedback mechanism to quantify the loss of information in the rendering process makes the design of good transfer functions a difficult and time consuming task. In this paper, we present the general notion of visibility histograms, which are multidimensional graphical representations of the distribution of visibility in a volume-rendered image. In this paper, we explore the 1D and 2D transfer functions that result from intensity values and gradient magnitude. With the help of these histograms, users can manage a complex set of transfer function parameters that maximize the visibility of the intervals of interest and provide high quality images of volume data. We present a semiautomated method for generating transfer functions, which progressively explores the transfer function space toward the goal of maximizing visibility of important structures. Our methodology can be easily deployed in most visualization systems and can be used together with traditional 1D and 2D opacity transfer functions based on scalar values, as well as with other more sophisticated rendering algorithms.

Are cortical models really bound by the "binding problem"?

Recently, 3D Gaussian Splatting (3DGS) has demonstrated impressive novel view synthesis results, while allowing the rendering of high-resolution images in real-time. However, leveraging 3D Gaussians for surface reconstruction poses significant challenges due to the explicit and disconnected nature of 3D Gaussians. In this work, we present Gaussian Opacity Fields (GOF), a novel approach for efficient, high-quality, and adaptive surface reconstruction in unbounded scenes. Our GOF is derived from ray-tracing-based volume rendering of 3D Gaussians, enabling direct geometry extraction from 3D Gaussians by identifying its levelset, without resorting to Poisson reconstruction or TSDF fusion as in previous work. We approximate the surface normal of Gaussians as the normal of the ray-Gaussian intersection plane, enabling the application of regularization that significantly enhances geometry. Furthermore, we develop an efficient geometry extraction method utilizing Marching Tetrahedra, where the tetrahedral grids are induced from 3D Gaussians and thus adapt to the scene's complexity. Our evaluations reveal that GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis. Further, it compares favorably to or even outperforms, neural implicit methods in both quality and speed.

The transfer function is crucial for direct volume rendering (DVR) to create an informative visual representation of volumetric data. However, manually adjusting the transfer function to achieve the desired DVR result can be time-consuming and unintuitive. In this paper, we propose Differentiable Design Galleries, an image-based transfer function design approach to help users explore the design space of transfer functions by taking advantage of the recent advances in deep learning and differentiable rendering. Specifically, we leverage neural rendering to learn a latent design space, which is a continuous manifold representing various types of implicit transfer functions. We further provide a set of interactive tools to support intuitive query, navigation, and modification to obtain the target design, which is represented as a neural-rendered design exemplar. The explicit transfer function can be reconstructed from the target design with a differentiable direct volume renderer. Experimental results on real volumetric data demonstrate the effectiveness of our method.

Three-Dimensional Gaussian Splatting (3DGS) has revolutionized novel-view synthesis, enabling real-time rendering of high-quality scenes. Two-Dimensional Gaussian Splatting (2DGS) improves geometric accuracy by replacing 3D Gaussians with flat 2D Gaussians. However, the flat nature of 2D Gaussians reduces mesh quality on volumetric surfaces and results in over-smoothed reconstruction. To address this, we propose Dual-Dimensional Gaussian Splatting (DDGS), which integrates both 2D and 3D Gaussians. First, we generalize the homogeneous transformation matrix based on 2DGS to initialize all Gaussians in 3D. Subsequently, during training, we selectively convert Gaussians into 2D representations based on their scale. This approach leverages the complementary strengths of 2D and 3D Gaussians, resulting in more accurate surface reconstruction across both flat and volumetric regions. Additionally, to mitigate over-smoothing, we introduce gradient-based regularization terms. Quantitative evaluations on the DTU and TnT datasets demonstrate that DDGS consistently outperforms prior methods, including 3DGS, SuGaR, and 2DGS, achieving the best Chamfer Distance and F1 score across a wide range of scenes.

The transfer function is the basis of direct volume rendering, and how to design it is also a crucial step for the volume visualization. Multi-dimensional transfer functions are a very effective way to extract materials and their boundaries for scalar data. For many applications, boundaries carry most of the relevant information. Based on the definition of transfer functions, this paper discusses the principle of the multidimensional transfer function based on boundaries which uses the scalar values and the gradient magnitudes as the domains, and presents a method of calculation the gradient magnitude, then implements a histogram of the occurrences of the multiple data values. Finally, we present a set of direct manipulation widgets that guide users towards transfer function settings appropriate for their visualization and data exploration interests. The transfer function and widgets combine to form a powerful system for interactive volume exploration.

<p>Over the last few years, the amount of large and complex data in the public domain has increased enormously and new challenges arose in the representation, analysis and visualization of such data. Considering the number of space missions that provided and will provide remote sensing data, there is still the need of a system that can be dispatched in several remote repositories and being accessible from a single client of commodity hardware.</p><p>To tackle this challenge, at the DLR Institute for Software Technology we have defined a dual backend frontend system, enabling the interactive analysis and visualization of large-scale remote sensing data. The basis for all visualization and interaction approaches is CosmoScout VR, a visualization tool developed internally at DLR, and publicly available on Github, that allows the visualization of complex planetary data and large simulation data in real-time. The dual component of this system is based on an MPI framework, called Viracocha, that enables the analysis of large data remotely, and allows the efficient network usage about sending compact and partial results for interactive visualization in CosmoScout as soon as they are computed.</p><p>A node-based interface is defined within the visualization tool, and this lets a domain expert to easily define customized pipelines for processing and visualizing the remote data. Each “node” of this interface is either linked with a feature extraction module, defined in Viracocha, or to a rendering module defined directly in CosmoScout. Being this interface completely customizable by a user, multiple pipelines can be defined over the same dataset to enhance even more the visualization feedback for analysis purposes.</p><p>Being an ongoing project, on top of these tools, as a novel strategy in EO data processing and visualization, we plan to define and implement strategies based on Topological Data Analysis (TDA). TDA is an emerging set of technique for processing the data considering its topological features. These include both the geometric information associated to a point, as well all the non-geometric scalar values, like temperature and pressure, to name a few, that can be captured during a monitoring mission. One of the major theories behind TDA is Discrete Morse Theory, that, given a scalar value, is used to define a gradient on such function, extract the critical points, identify the region-of-influence of each critical point, and so on. This strategy is parameter free and enables a domain scientist to process large datasets without a prior knowledge of it.</p><p>An interesting research question, that it will be investigated during this project is the correlation of changes of critical points at different time steps, and the identification of deformation (or changes) across time in the original dataset.</p>