Scientific Visualization Research Articles

Enhancing Text-to-Image Generation: Integrating CLIP and Diffusion Models for Improved Visual Accuracy and Semantic Consistency

Abstract. Text-to-Image (T2I) generation focuses on producing images that precisely match given textual descriptions by combining techniques from computer vision and natural language processing (NLP). Existing studies have shown an innovative approach to enhance T2I generation by integrating Contrastive Language-Image Pretraining (CLIP) embeddings with a Diffusion Model (DM). The method involves initially extracting rich and meaningful text embeddings using CLIP, which are then transformed into corresponding images. These images are progressively refined through an iterative denoising process enabled by diffusion models. Comprehensive experiments conducted on the MS-COCO dataset validate the proposed method, demonstrating significant improvements in image fidelity and the alignment between text and images. When compared to traditional models like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), which often struggle with maintaining both visual quality and semantic accuracy, this hybrid model shows superior performance. Future research could explore optimizing hybrid models further and applying T2I technology to specialized fields, such as medical imaging and scientific visualization, expanding its potential use cases.

Appealing, but misleading: a warning against a naive AI realism

AbstractScientists, educators, and instructional designers are facing numerous challenges due to the introduction of generative AI tools that can create appealing realistic imagery based on text prompts. Given that realism contributes to the trustworthiness of images coupled with people’s eagerness to externalize complex tasks to AI systems, the problem of a naive AI realism arises in which image creation and optimization is offloaded without considering the limitations of AI-driven technology. However, scientific visualizations and images used for educational purposes must go beyond an appealing presentation; above all, they should be accurate and factually correct. We argue that the utilization of generative AI tools for these types of visualizations requires human oversight, subject matter knowledge, and knowledge of effective design. In particular, we warn of a naive adoption of technological possibilities to “optimize” visualizations for educational purposes, such as memorability. A reductionist and naive view of AI-based optimization that fails to take into account the complex prerequisites for learning and instruction is likely to have negative consequences.

Development trends and knowledge framework of artificial intelligence (AI) applications in oncology by years: a bibliometric analysis from 1992 to 2022

PurposeOncology is the primary field in medicine with a high rate of artificial intelligence (AI) use. Thus, this study aimed to investigate the trends of AI in oncology, evaluating the bibliographic characteristics of articles. We evaluated the related research on the knowledge framework of Artificial Intelligence (AI) applications in Oncology through bibliometrics analysis and explored the research hotspots and current status from 1992 to 2022.MethodsThe research employed a scientometric methodology and leveraged scientific visualization tools such as Bibliometrix R Package Software, VOSviewer, and Litmaps for comprehensive data analysis. Scientific AI-related publications in oncology were retrieved from the Web of Science (WoS) and InCites from 1992 to 2022.ResultsA total of 7,815 articles authored by 35,098 authors and published in 1,492 journals were included in the final analysis. The most prolific authors were Esteva A (citaition = 5,821) and Gillies RJ (citaition = 4288). The most active institutions were the Chinese Academy of Science and Harward University. The leading journals were Frontiers ın Oncology and Scientific Reports. The most Frequent Author Keywords are "machine learning", "deep learning," "radiomics", "breast cancer", “melanoma” and "artificial intelligence," which are the research hotspots in this field. A total of 10,866 Authors' keywords were investigated. The average number of citations per document is 23. After 2015, the number of publications proliferated.ConclusionThe investigation of Artificial Intelligence (AI) applications in the field of Oncology is still in its early phases especially for genomics, proteomics, and clinicomics, with extensive studies focused on biology, diagnosis, treatment, and cancer risk assessment. This bibliometric analysis offered valuable perspectives into AI's role in Oncology research, shedding light on emerging research paths. Notably, a significant portion of these publications originated from developed nations. These findings could prove beneficial for both researchers and policymakers seeking to navigate this field.

A Generative Adversarial Network for Upsampling of Direct Volume Rendering Images

AbstractDirect volume rendering (DVR) is an important tool for scientific and medical imaging visualization. Modern GPU acceleration has made DVR more accessible; however, the production of high‐quality rendered images with high frame rates is computationally expensive. We propose a deep learning method with a reduced computational demand. We leveraged a conditional generative adversarial network (cGAN) to upsample DVR images (a rendered scene), with a reduced sampling rate to obtain similar visual quality to that of a fully sampled method. Our dvrGAN is combined with a colour‐based loss function that is optimized for DVR images where different structures such as skin, bone, etc. are distinguished by assigning them distinct colours. The loss function highlights the structural differences between images, by examining pixel‐level colour, and thus helps identify, for instance, small bones in the limbs that may not be evident with reduced sampling rates. We evaluated our method in DVR of human computed tomography (CT) and CT angiography (CTA) volumes. Our method retained image quality and reduced computation time when compared to fully sampled methods and outperformed existing state‐of‐the‐art upsampling methods.

Fast Comparative Analysis of Merge Trees Using Locality Sensitive Hashing.

Scalar field comparison is a fundamental task in scientific visualization. In topological data analysis, we compare topological descriptors of scalar fields-such as persistence diagrams and merge trees-because they provide succinct and robust abstract representations. Several similarity measures for topological descriptors seem to be both asymptotically and practically efficient with polynomial time algorithms, but they do not scale well when handling large-scale, time-varying scientific data and ensembles. In this paper, we propose a new framework to facilitate the comparative analysis of merge trees, inspired by tools from locality sensitive hashing (LSH). LSH hashes similar objects into the same hash buckets with high probability. We propose two new similarity measures for merge trees that can be computed via LSH, using new extensions to Recursive MinHash and subpath signature, respectively. Our similarity measures are extremely efficient to compute and closely resemble the results of existing measures such as merge tree edit distance or geometric interleaving distance. Our experiments demonstrate the utility of our LSH framework in applications such as shape matching, clustering, key event detection, and ensemble summarization.

Ten Years of Operating a Center with Large-Scale Virtual-Reality Installations: Developments and Learnings

Abstract Virtual reality (VR) can be found in many fields including art history, education, research, and industry 4.0. Next to lightweight head-mounted displays there are also VR installations, such as immersive 3D screens, large-scale displays, and CAVE systems, that are used in research, scientific visualization, and also the automotive industry. These systems offer high visual quality and collaborative VR experiences for researchers and have long been used in research. We present learnings and insights from ten years of operating and maintaining a visualization center with large-scale immersive displays and installations. Our report aims to answer questions on the benefits and challenges of such a location-based VR center. We broke down the ten years into three phases and discuss the installations themselves, software, and other various developments of the center over time, illustrated with exemplary use cases. Finally, we cover our experiences at the beginning of phase four with the installation of a (novel) LED CAVE and the future (forward-looking) developments we expect for location-based VR centers.

ColocZStats: a z-stack signal colocalization extension tool for 3D slicer.

Confocal microscopy has evolved to be a widely adopted imaging technique in molecular biology and is frequently utilized to achieve accurate subcellular localization of proteins. Applying colocalization analysis on image z-stacks obtained from confocal fluorescence microscopes is a dependable method of revealing the relationship between different molecules. In addition, despite the established advantages and growing adoption of 3D visualization software in various microscopy research domains, there have been few systems that can support colocalization analysis within a user-specified region of interest (ROI). In this context, several broadly employed biological image visualization platforms are meticulously explored in this study to understand the current landscape. It has been observed that while these applications can generate three-dimensional (3D) reconstructions for z-stacks, and in some cases transfer them into an immersive virtual reality (VR) scene, there is still little support for performing quantitative colocalization analysis on such images based on a user-defined ROI and thresholding levels. To address these issues, an extension called ColocZStats (pronounced Coloc-Zee-Stats) has been developed for 3D Slicer, a widely used free and open-source software package for image analysis and scientific visualization. With a custom-designed user-friendly interface, ColocZStats allows investigators to conduct intensity thresholding and ROI selection on imported 3D image stacks. It can deliver several essential colocalization metrics for structures of interest and produce reports in the form of diagrams and spreadsheets.

XR-based interactive visualization platform for real-time exploring dynamic earth science data

The transition from 2D planar displays to immersive holographic 3D environments has brought advancements in visualization technology. However, there remains a lack of effective interactive visualization tools for complex multi-dimensional structured or unstructured datasets in immersive space. To address this gap, we have developed MetIVA, a state-of-the-art multiscale interactive data visualization platform that leverages the Extended Reality (XR) and cloud rendering technology for immersive data exploration. In this paper, we firstly outline the historical development of scientific visualization and the recent shift towards 3D and higher-dimensional visualization, and then basically introduce the conceptual framework and platform structure of MetIVA, and finally present the evaluation results from recruited potential users. The results confirm that MetIVA is a powerful tool to accelerate data exploration and decision-making processes. Its interactive and intuitive features, along with ongoing optimization efforts, make it a valuable tool for researchers and practitioners in the field of Earth science.

2023 IEEE Scientific Visualization Contest Winner: VisAnywhere: Developing Multi-platform Scientific Visualization Applications.

Scientists often explore and analyze large-scale scientific simulation data by leveraging two- and three-dimensional visualizations. The data and tasks can be complex and therefore best supported using myriad display technologies, from mobile devices to large high-resolution display walls to virtual reality headsets. Using a simulation of neuron connections in the human brain provided for the 2023 IEEE Scientific Visualization Contest, we present our work leveraging various web technologies to create a multi-platform scientific visualization application. Users can spread visualization and interaction across multiple devices to support flexible user interfaces and both co-located and remote collaboration. Drawing inspiration from responsive web design principles, this work demonstrates that a single codebase can be adapted to develop scientific visualization applications that operate everywhere.

Visualization at exascale: Making it all work with VTK-m

The VTK-m software library enables scientific visualization on exascale-class supercomputers. Exascale machines are particularly challenging for software development in part because they use GPU accelerators to provide the vast majority of their computational throughput. Algorithmic designs for GPUs and GPU-centric computing often deviate from those that worked well on previous generations of high-performance computers that relied on traditional CPUs. Fortunately, VTK-m provides scientific visualization algorithms for GPUs and other accelerators. VTK-m also provides a framework that simplifies the implementation of new algorithms and adds a porting layer to work across multiple processor types. This paper describes the main challenges encountered when making scientific visualization available at exascale. We document the surprises and obstacles faced when moving from pre-exascale platforms to the final exascale designs and the performance on those systems including scaling studies on Frontier, an exascale machine with over 37,000 AMD GPUs. We also report on the integration of VTK-m with other exascale software technologies. Finally, we show how VTK-m helps scientific discovery for applications such as fusion and particle acceleration that leverage an exascale supercomputer.

PredATW: Predicting the Asynchronous Time Warp Latency For VR Systems

With the advent of low-power ultra-fast hardware and GPUs, virtual reality (VR) has gained a lot of prominence in the past few years and is being used in various areas, such as education, entertainment, scientific visualization, and computer-aided design. VR-based applications are highly interactive, and one of the most important performance metrics for these applications is the motion-to-photon-delay (MPD). MPD is the delay from the user’s head movement to the time at which the image gets updated on the VR screen. Since the human visual system can even detect an error of a few pixels (very spatially sensitive), the MPD should be as small as possible. Popular VR vendors use the GPU-accelerated Asynchronous Time Warp (ATW) algorithm to reduce the MPD. ATW reduces the MPD if and only if the warping operation finishes just before the display refreshes. However, due to the competition between the different constituent applications for the single, shared GPU, the GPU-accelerated ATW algorithm suffers from an unpredictable ATW latency, making it challenging to find the ideal time instance for starting the time warp and ensuring that it completes with the least amount of lag relative to the screen refresh. Hence, the state-of-the-art is to use a separate hardware unit for the time-warping operation. Our approach, PredATW , uses an ML-based hardware predictor to predict the ATW latency for a VR application, and then schedule it as late as possible while running the time-warping operation on the GPU itself. As far as we know, this is the first work to do so. Our predictor achieves an error of only 0.22 ms across several popular VR applications for predicting the ATW latency. As compared to the baseline architecture, we reduce deadline misses by 80.6%.

Mint: Integrating scientific visualizations into virtual reality

We present an image-based approach to integrate state-of-the-art scientific visualization into virtual reality (VR) environments: the mint visualization/VR inter-operation system. We enable the integration of visualization algorithms from within their software frameworks directly into VR without the need to explicitly port visualization implementations to the underlying VR framework—thus retaining their capabilities, specializations, and optimizations. Consequently, our approach also facilitates enriching VR-based scientific data exploration with established or novel VR immersion and interaction techniques available in VR authoring tools. The separation of concerns enables researchers and users in different domains, like virtual immersive environments, immersive analytics, and scientific visualization, to independently work with existing software suitable for their domain while being able to interface with one another easily. We present our system architecture and inter-operation protocol (mint), an example of a collaborative VR environment implemented in the Unity engine (VRAUKE), as well as the integration of the protocol for the visualization frameworks Inviwo, MegaMol, and ParaView. Our implementation is publicly available as open-source software.Graphical abstract

Variational Feature Extraction in Scientific Visualization

Across many scientific disciplines, the pursuit of even higher grid resolutions leads to a severe scalability problem in scientific computing. Feature extraction is a commonly chosen approach to reduce the amount of information from dense fields down to geometric primitives that further enable a quantitative analysis. Examples of common features are isolines, extremal lines, or vortex corelines. Due to the rising complexity of the observed phenomena, or in the event of discretization issues with the data, a straightforward application of textbook feature definitions is unfortunately insufficient. Thus, feature extraction from spatial data often requires substantial pre- or post-processing to either clean up the results or to include additional domain knowledge about the feature in question. Such a separate pre- or post-processing of features not only leads to suboptimal and incomparable solutions, it also results in many specialized feature extraction algorithms arising in the different application domains. In this paper, we establish a mathematical language that not only encompasses commonly used feature definitions, it also provides a set of regularizers that can be applied across the bounds of individual application domains. By using the language of variational calculus, we treat features as variational minimizers, which can be combined and regularized as needed. Our formulation not only encompasses existing feature definitions as special case, it also opens the path to novel feature definitions. This work lays the foundations for many new research directions regarding formal definitions, data representations, and numerical extraction algorithms.

A New Default Colormap for ParaView.

ParaView is one of the most prominent software tools for scientific visualization used by scientists around the world. Color is a primary conduit to visually map data to its representation and, thus, enable investigation and interpretation of the data. Colormap selection has a significant impact on the data revealed; its design and selection is a critical aspect of scientific data visualization. A common choice for a user is the program's default colormap, so careful consideration of this default is consequential. Although the current default colormap in ParaView, a succession of hues from cool blue to warm red, has served the community well, research shows that more nuanced colormap configurations increase discriminability while maintaining other critical metrics. These findings inspire us to revisit and update the default colors in ParaView. Here we present a new ParaView default colormap, the criteria and methods of development, and example visualizations and analytic metrics.

MEGA Vision: Integrating Reef Photogrammetry Data into Immersive Mixed Reality Experiences

Abstract. Coral reefs and submerged cultural heritage sites are integral to supporting marine biodiversity, preserving human history, providing ecosystem services, and understanding drivers of ecosystem health and function. Despite the importance of these submerged underwater habitats, accessibility to these environments remains limited to specialized professionals. The MEGA Vision mixed reality application integrates photogrammetry-derived data products with augmented reality (AR) technologies to transcend this barrier, offering an immersive and educational platform for the broader public. Using high-resolution imagery from SCUBA expeditions, the app presents users with realistic and spatially accurate 3D reconstructions of coral reefs and submerged archaeological artifacts within an interactive interface developed through Unity and Vuforia. The applications’ instructional design includes multimedia elements for enhancing user comprehension of marine and historical sciences. This mixed reality tool exemplifies the convergence of scientific data visualization and public engagement, offering a unique educational tool that demystifies the complexities of marine ecosystems and maritime history, thereby fostering a deeper appreciation and stewardship of underwater environments. By enabling accessible, interactive, and immersive experiences, the application has the potential to revolutionize the way we interact with and contribute to marine sciences, aligning technology with conservation and research efforts to cultivate a more informed and environmentally conscious public.

Beyond ExaBricks: GPU Volume Path Tracing of AMR Data

AbstractAdaptive Mesh Refinement (AMR) is becoming a prevalent data representation for HPC, and thus also for scientific visualization. AMR data is usually cell centric (which imposes numerous challenges), complex, and generally hard to render. Recent work on GPU‐accelerated AMR rendering has made much progress towards real‐time volume and isosurface rendering of such data, but so far this work has focused exclusively on ray marching, with simple lighting models and without scattering events or global illumination. True high‐quality rendering requires a modified approach that is able to trace arbitrary incoherent paths; but this may not be a perfect fit for the types of data structures recently developed for ray marching. In this paper, we describe a novel approach to high‐quality path tracing of complex AMR data, with a specific focus on analyzing and comparing different data structures and algorithms to achieve this goal.

Freak Pictures, Fly Fishers, and Optical Illusions: Historical Antecedents of the Science of Fish Vision

Abstract In the early twentieth century, when little was known about the underwater realm, the question of subaquatic vision garnered much interest. Based on human laws of optics in water, observers of the time claimed to show how fish see, and the impact of illusions on fish vision. Addressing fly-fishers in particular, they raised the question of whether fish see “freak pictures” and “monsters.” The present study examines five scientific visualizations published in Scientific American in 1913 to determine whether optical illusions remain relevant for fish scientists today and whether conceptions of visual intelligence in fish have changed since 1913.

Seeing what you believe: recognition memory for evolutionary tree structure is affected by students’ misconceptions

ABSTRACT Peoples’ recognition memory for pictorial stimuli is extremely good. Even complex scientific visualisations are recognised with a high degree of accuracy. The present research examined recognition memory for the branching structure of evolutionary trees. This is an educationally consequential topic due to the potential for contamination from students’ misconceptions. The authors created six pairs of scientifically accurate and structurally identical evolutionary trees that differed in whether they included a taxon that cued a misconception in memory. As predicted, Experiment 1 found that (a) college students (N = 90) had better memory for each of the six tree structures when a neutral taxon (M = 0.73) rather than a misconception-cuing taxon (M = 0.64) was included in the tree, and (b) recognition memory was significantly above chance for both sets of trees. Experiment 2 ruled out an alternative hypothesis based on the possibility that 8–12 sec was not enough time for students to encode the relationships depicted in the trees. The authors consider implications of these results for using evolutionary trees to better communicate scientific information. This is important because these trees provide information that is relevant for everyday life.

Empirical research on banks' risk disclosure: Systematic literature review, bibliometric analysis and future research agenda

The purpose of this paper is to outline the current state of empirical research on banks' risk reporting. In addition to the development of the research field over time, regulatory trends and drivers for academic research on risk reporting will be derived. The review follows a triangulated approach: In addition to a qualitative content analysis based on the SLR, a quantitative bibliometric analysis using the scientific visualization techniques bibliometrix and VOSviewer will serve as a robustness check. The sole focus of this SLR on banks reflects the regulatory specificity of the financial services industry and serves to derive recommendations for action for regulators, supervisors, and auditors. The article follows the tenor of macro- and micro-prudential banking regulation, which has raised market discipline to a new level through the implementation of regulatory disclosure standards in the context of the Basel III amendment, with a deliberate clustering of the research area according to risk types. By identifying research gaps and conceptualizing a research agenda, this paper continues to serve the academia to broaden the research field of risk disclosure, especially for banks.

Navigating color integrity in data visualization

Color is crucial in scientific visualization, yet it is often misused. Addressing this, we think accessible and accurate techniques, such as color-blind friendly palettes and perceptually even gradients, are vital. Accountability and basic knowledge in data visualization are key in fostering a culture of color integrity, ensuring accurate and inclusive data representation.