Real-time Interactive Visualization Research Articles

Developing a patient flow visualization and prediction model using aggregated data for a healthcare network cluster in Southwest Ethiopia.

A health information system has been created to gather, aggregate, analyze, interpret, and utilize data collected from diverse sources. In Ethiopia, the most popular digital tools are the Electronic Community Health Information System and the District Health Information System. However, these systems lack capabilities like real-time interactive visualization and a data-driven engine for evidence-based insights. As a result, it was challenging to observe and continuously monitor the flow of patients. To address the gap, this study used aggregated data to visualize and predict patient flow in a South Western Ethiopia healthcare network cluster. The South-Western Ethiopian healthcare network cluster was where the patient flow datasets were collected. The collected dataset encompasses a span of 41 months, from 2019 to 2022, and has been obtained from 21 hospitals and health centers. Python Sankey diagrams were used to develop and build patient flow visualizations. Then, using the random forest and K-Nearest Neighbors (KNN) algorithms, we achieved an accuracy of 0.85 and 0.83 for the outpatient flow modeling and prediction, respectively. The imbalance in the data was further addressed using the NearMiss Algorithm, Synthetic Minority Oversampling Technique (SMOTE), and SMOTE-Tomek methods. In conclusion, we developed a patient flow visualization and prediction model as a first step toward an end-to-end effective real-time patient flow data-driven and analytical dashboard in Ethiopia, as well as a plugin for the already-existing digital health information system. Moreover, the need for and amount of data created by these digital tools will grow along with their use, demanding effective data-driven visualization and prediction to support evidence-based decision-making.

Visualization of isomorphism-synesthesia of colour and music

Music and colour, as human hearing and visual art, are closely related to human psychological feelings and symbolic associations. There is an isomorphic relationship between music and colour. The article uses the concept of “synesthesia” in psychology and the “co-construction” relationship in mathematics as a bridge, based on Kandinsky’s “inner sound” theory and Mallion’s “tone-colour system”, an interdisciplinary theoretical model of “timbre isomorphism synesthesia” (ISCM) is constructed. At the practical level, based on the ISCM theory, a set of timbre synesthesia visualization tools ASAH and visualization processes are designed, through music data input, graphics mapping visualization, colour mapping visualization, real-time interactive visualization, and finally output timbre synesthesia visualization works. In order to avoid the visual homogenization caused by algorithm design, ASHA has set up a custom editor, which emphasizes the individual differences and multi-sensory experience of tonal synesthesia visualization.

Real-Time Interactive Parallel Visualization of Large-Scale Flow-Field Data

With the increasing demand for high precision in numerical simulations using computational fluid dynamics (CFD), the use of large-scale grids for discretized solutions has become a trend, resulting in an explosive growth of flow-field data size. To address the challenges posed by large-scale flow-field data for real-time interactive visualization, this paper proposes novel strategies for data partitioning and communication management. Firstly, we propose a data-partitioning strategy based on grid segmentation. This approach constructs metadata to create file viewports for each process and performs grid partitioning. Subsequently, it reconstructs sub-grids within each process and utilizes a coordinate-mapping algorithm to map global coordinates to local process coordinates, facilitating access to attribute variables through a lookup table. Secondly, we introduce a real-time interactive method for large-scale flow fields. This method leverages the system architecture of high-speed interconnection among compute nodes in a cluster environment and low-speed interconnection between service nodes and rendering nodes. It enables coordinated management of parallel rendering and synchronized rendering methods. The experimental results on typical flow-field data demonstrate that the proposed data-partitioning strategy improves the loading speed of millions of grid-level data by a factor of 7, surpassing ParaView’s performance by 1.5 times. Furthermore, it achieves system load balancing. Real-time interaction experiments with datasets containing 500 million and 800 million grid cells exhibit millisecond-level latencies, demonstrating the effectiveness of the proposed communication management method in meeting the real-time interactive visualization demands of large-scale flow-field data.

NetBID2 provides comprehensive hidden driver analysis

Many signaling and other genes known as “hidden” drivers may not be genetically or epigenetically altered or differentially expressed at the mRNA or protein levels, but, rather, drive a phenotype such as tumorigenesis via post-translational modification or other mechanisms. However, conventional approaches based on genomics or differential expression are limited in exposing such hidden drivers. Here, we present a comprehensive algorithm and toolkit NetBID2 (data-driven network-based Bayesian inference of drivers, version 2), which reverse-engineers context-specific interactomes and integrates network activity inferred from large-scale multi-omics data, empowering the identification of hidden drivers that could not be detected by traditional analyses. NetBID2 has substantially re-engineered the previous prototype version by providing versatile data visualization and sophisticated statistical analyses, which strongly facilitate researchers for result interpretation through end-to-end multi-omics data analysis. We demonstrate the power of NetBID2 using three hidden driver examples. We deploy NetBID2 Viewer, Runner, and Cloud apps with 145 context-specific gene regulatory and signaling networks across normal tissues and paediatric and adult cancers to facilitate end-to-end analysis, real-time interactive visualization and cloud-based data sharing. NetBID2 is freely available at https://jyyulab.github.io/NetBID.

Organizing and visualizing point clouds with continuous levels of detail

Point clouds contain high detail and high accuracy geometry representation of the scanned Earth surface parts. To manage the huge amount of data, the point clouds are traditionally organized on location and map-scale; e.g. in an octree structure, where top-levels of the tree contain few points suitable for small scale overviews and lower levels of the tree contain more points suitable for large scale detailed views. The drawback of this solution is that it is based on discrete levels, causing visual artifacts in the form of data density shocks when creating the commonly used perspective views. This paper presents a method based on an optimized distribution of points over continuous levels, avoiding the visualization shocks. The traditional distribution ratio’s of data amounts over discrete levels of raster or vector data is considered the reference. How to convert this to point clouds with continuous levels (still benefiting from the proven advantages of the data distribution in discrete levels for efficient access at a wide range of scales)? In our solution, for each point a cLoD (continuous Level of Detail) value is computed and added as dimension to the point. A SFC (Space Filling Curve)-based nD data clustering technique can be used to organize the points, so that they can be efficiently queried. It should be noted that also other multi-dimensional indexing and clustering techniques could be applied to realize continuous levels based on the cLoD value. Besides the mathematical foundation of the approach also several implementations are described, varying from a 3D web-browser based solution to an augmented reality point cloud app in a mobile phone. The cLoD enables interactive real-time visualization using perspective views without data density shocks, while supporting continuous zoom-in/out and progressive data streaming between server and client. The described cLoD based approach is generic and supports different types of point clouds: from airborne, terrestrial, mobile and indoor laser scanning, but also from dense matching optical imagery or multi-beam echo soundings.

MicrobioSee: A Web-Based Visualization Toolkit for Multi-Omics of Microbiology.

With the upgrade and development of the high-throughput sequencing technology, multi-omics data can be obtained at a low cost. However, mapping tools that existed for microbial multi-omics data analysis cannot satisfy the needs of data description and result in high learning costs, complex dependencies, and high fees for researchers in experimental biology fields. Therefore, developing a toolkit for multi-omics data is essential for microbiologists to save effort. In this work, we developed MicrobioSee, a real-time interactive visualization tool based on web technologies, which could visualize microbial multi-omics data. It includes 17 modules surrounding the major omics data of microorganisms such as the transcriptome, metagenome, and proteome. With MicrobioSee, methods for plotting are simplified in multi-omics studies, such as visualization of diversity, ROC, and enrichment pathways for DEGs. Subsequently, three case studies were chosen to represent the functional application of MicrobioSee. Overall, we provided a concise toolkit along with user-friendly, time-saving, cross-platform, and source-opening for researchers, especially microbiologists without coding experience. MicrobioSee is freely available at https://microbiosee.gxu.edu.cn.

Real-time interactive visualization of large networks on a tiled display system

This paper introduces a methodology for visualizing large real-world (social) network data on a high-resolution tiled display system. Advances in network drawing algorithms enabled real-time visualization and interactive exploration of large real-world networks. However, visualization on a typical desktop monitor remains challenging due to the limited amount of screen space and ever increasing size of real-world datasets.To solve this problem, we propose an integrated approach that employs state-of-the-art network visualization algorithms on a tiled display system consisting of multiple screens. Key to our approach is to use the machine’s graphics processing units (GPUs) to their fullest extent, in order to ensure an interactive setting with real-time visualization. To realize this, we extended a recent GPU-based implementation of a force-directed graph layout algorithm to multiple GPUs and combined this with a distributed rendering approach in which each graphics card in the tiled display system renders precisely the part of the network to be displayed on the monitors attached to it.Our evaluation of the approach on a 12-screen 25 megapixels tiled display system with three GPUs, demonstrates interactive performance at 60 frames per second for real-world networks with tens of thousands of nodes and edges. This constitutes a performance improvement of approximately 4 times over a single GPU implementation. All the software developed to implement our tiled visualization approach, including the multi-GPU network layout, rendering, display and interaction components, are made available as open-source software.

Improved interactive color visualization approach for hyperspectral images

Hyperspectral images (HSIs) have become increasingly prominent as they can maintain the subtle spectral differences of the imaged objects. Designing approaches and tools for analyzing HSIs presents a unique set of challenges due to their high-dimensional characteristics. An improved color visualization approach is proposed in this article to achieve communication between users and HSIs in the field of remote sensing. Under the real-time interactive control and color visualization, this approach can help users intuitively obtain the rich information hidden in original HSIs. Using the dimensionality reduction (DR) method based on band selection, high-dimensional HSIs are reduced to low-dimensional images. Through drop-down boxes, users can freely specify images that participate in the combination of RGB channels of the output image. Users can then interactively and independently set the fusion coefficient of each image within an interface based on concentric circles. At the same time, the output image will be calculated and visualized in real time, and the information it reflects will also be different. In this approach, channel combination and fusion coefficient setting are two independent processes, which allows users to interact more flexibly according to their needs. Furthermore, this approach is also applicable for interactive visualization of other types of multi-layer data.

Computational Analysis and Phylogenetic Clustering of SARS-CoV-2 Genomes.

COVID-19, the disease caused by the novel SARS-CoV-2 coronavirus, originated as an isolated outbreak in the Hubei province of China but soon created a global pandemic and is now a major threat to healthcare systems worldwide. Following the rapid human-to-human transmission of the infection, institutes around the world have made efforts to generate genome sequence data for the virus. With thousands of genome sequences for SARS-CoV-2 now available in the public domain, it is possible to analyze the sequences and gain a deeper understanding of the disease, its origin, and its epidemiology. Phylogenetic analysis is a potentially powerful tool for tracking the transmission pattern of the virus with a view to aiding identification of potential interventions. Toward this goal, we have created a comprehensive protocol for the analysis and phylogenetic clustering of SARS-CoV-2 genomes using Nextstrain, a powerful open-source tool for the real-time interactive visualization of genome sequencing data. Approaches to focus the phylogenetic clustering analysis on a particular region of interest are detailed in this protocol.

FastPathology: An Open-Source Platform for Deep Learning-Based Research and Decision Support in Digital Pathology

Deep convolutional neural networks (CNNs) are the current state-of-the-art for digital analysis of histopathological images. The large size of whole-slide microscopy images (WSIs) requires advanced memory handling to read, display and process these images. There are several open-source platforms for working with WSIs, but few support deployment of CNN models. These applications use third-party solutions for inference, making them less user-friendly and unsuitable for high-performance image analysis. To make deployment of CNNs user-friendly and feasible on low-end machines, we have developed a new platform, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FastPathology</i> , using the FAST framework and C++. It minimizes memory usage for reading and processing WSIs, deployment of CNN models, and real-time interactive visualization of results. Runtime experiments were conducted on four different use cases, using different architectures, inference engines, hardware configurations and operating systems. Memory usage for reading, visualizing, zooming and panning a WSI were measured, using FastPathology and three existing platforms. FastPathology performed similarly in terms of memory to the other C++-based application, while using considerably less than the two Java-based platforms. The choice of neural network model, inference engine, hardware and processors influenced runtime considerably. Thus, FastPathology includes all steps needed for efficient visualization and processing of WSIs in a single application, including inference of CNNs with real-time display of the results. Source code, binary releases, video demonstrations and test data can be found online on GitHub at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/SINTEFMedtek/FAST-Pathology/</uri> .

An Overview about Research on 3D Visualization Technology of Geographic Information Based on Computer Software Development

Geographic Information System (GIS) is an important system of three-dimensional (3D) real-time interactive visualization platform. 3D visualization technology is to display the information to be expressed by a visual effect. The introduction of 3D visualization technology into GIS is a reform of the form of information expression, which can solve the limitation of using graphic symbols to express the 3D space world on a 2D plane. However, most of the visualizations in GIS focus on the 3D terrain visualization based on the surface model, which does not involve the volume visualization of spatial volume data. An overview about visualization technology for geographic information based on computer software is given here, to encourage further development.

Design of Interactive Visualizations for Next-Generation Ultra-Large Communication Networks

With the increasing size and complexity of next-generation communication networks, it is critical to utilize interactive visualizations to support the monitoring, planning, and management of networks. Effectively visualizing large-scale networks is difficult with traditional methods because of the high link density and complex node relationships. Given the limited screen space, to assist Internet Service Provider’s (ISP) network planning and management activities, investigating how to present ultra-large-scale network data efficiently is crucial. This paper presents a real-time interactive visualization system that combines the design strategies of progressive disclosure and multiple panels to elegantly visualize the large-scale networks and avoid the information-overload problem. The system also visualizes the configuration of the network elements and provides the network performance information, including the port-level Quality of Service (QoS) metrics. Furthermore, the system enables navigation through the port-level connection and provides different modes for multiple purposes.

Compression and Rendering of Space Environment Volume Data Based on Improved HVQ Algorithm

We propose an improved HVQ approach by combining two detail levels to compress space environment volumes, based on the variation characteristics including smooth variation and significantly positive correlation between variations in the same direction at two different scales. First, the space environment data is divided into 43 blocks. Then blocks are decomposed into a two-level hierarchical representation and each block is represented by a mean value and a detail vector. Finally, the detail vectors are encoded by a vector quantizer. During the vector quantization process, PCA-split is applied to compute an initial codebook, and then LBG-algorithm is conducted for codebook refinement and quantization. We take advantage of the codebook-retraining method to speed up the quantization of time series with temporal coherence. Furthermore, we employ the progressive rendering based on GPU for realtime interactive visualization. The results of our experiments prove that the algorithm proposed in this paper can significantly improve the compression rate and decoding speed without sacrificing fidelity in space environment domain.

Abstract 5117: Comprehensive structural analysis of cancer genomes by genome mapping

Abstract Tumors are often comprised of heterogeneous populations of cells, with certain cancer-driving mutations at low allele fractions in early stages of cancer development. Effective detection of such variants is critical for diagnosis and targeted treatment. However, typical short sequence reads are limited in their ability to span across repetitive regions of the genome and to facilitate structural variant (SV) analysis. Based on specific labeling and mapping of ultra-high molecular weight (UHMW) DNA, we developed a single-molecule platform that has the potential to detect disease-relevant SVs and give a high-resolution view of tumor heterogeneity. We developed a DNA isolation and sample preparation workflow that preserves the DNA integrity and conserves structural variation information from blood, cells and preserved tissue. Single molecules are labeled at specific motifs and analyzed in massively parallel nanochannels. The single-molecule maps are used in a bioinformatics pipeline that effectively detects structural variants at low allele fractions. It includes single-molecule based SV calling and fractional copy number analysis. Preliminary analyses using simulated and well-characterized cancer samples showed high sensitivity for variants of different types at as low as 5% allele fractions. The candidate variants are then annotated and further prioritized based on control data and publicly available annotations. The data are imported into a graphical user interface tool that includes new visualization tools (such as Circos diagrams) for real-time interactive visualization and curation. Bionano offers sample preparation, DNA imaging and genomic data analysis technologies combined into one streamlined workflow that enables high-throughput genome mapping on the Bionano Saphyr system. Together, these components allows for efficient analysis of any genome of interest. Citation Format: Ernest Lam, Yannick Delpu. Comprehensive structural analysis of cancer genomes by genome mapping [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 5117.

QuickRNASeq: Guide for Pipeline Implementation and for Interactive Results Visualization.

Sequencing of transcribed RNA molecules (RNA-Seq) has been used wildly for studying cell transcriptomes in bulk or at the single-cell level (Wang et al., Nat Rev Genet, 10:57-63, 2009; Ozsolak and Milos, Nat Rev Genet, 12:87-98, 2011; Sandberg, Nat Methods, 11:22-24, 2014) and is becoming the de facto technology for investigating gene expression level changes in various biological conditions, on the time course, and under drug treatments. Furthermore, RNA-Seq data helped identify fusion genes that are related to certain cancers (Maher et al., Nature, 458:97-101, 2009). Differential gene expression before and after drug treatments provides insights to mechanism of action, pharmacodynamics of the drugs, and safety concerns (Dixit et al., Genomics, 107:178-188, 2016). Because each RNA-Seq run generates tens to hundreds of millions of short reads with size ranging from 50 to 200bp, a tool that deciphers these short reads to an integrated and digestible analysis report is in high demand. QuickRNASeq (Zhao et al., BMC Genomics, 17:39-53, 2016) is an application for large-scale RNA-Seq data analysis and real-time interactive visualization of complex data sets. This application automates the use of several of the best open-source tools to efficiently generate user friendly, easy to share, and ready to publish report. Figures in this protocol illustrate some of the interactive plots produced by QuickRNASeq. The visualization features of the application have been further improved since its first publication in early 2016. The original QuickRNASeq publication (Zhao et al., BMC Genomics, 17:39-53, 2016) provided details of background, software selection, and implementation. Here, we outline the steps required to implement QuickRNASeq in user's own environment, as well as demonstrate some basic yet powerful utilities of the advanced interactive visualization modules in the report.

Celeris: A GPU-accelerated open source software with a Boussinesq-type wave solver for real-time interactive simulation and visualization

In this paper, we introduce an interactive coastal wave simulation and visualization software, called Celeris. Celeris is an open source software which needs minimum preparation to run on a Windows machine. The software solves the extended Boussinesq equations using a hybrid finite volume–finite difference method and supports moving shoreline boundaries. The simulation and visualization are performed on the GPU using Direct3D libraries, which enables the software to run faster than real-time. Celeris provides a first-of-its-kind interactive modeling platform for coastal wave applications and it supports simultaneous visualization with both photorealistic and colormapped rendering capabilities. We validate our software through comparison with three standard benchmarks for non-breaking and breaking waves. Program summaryProgram title: CelerisProgram Files doi:http://dx.doi.org/10.17632/5djwvf5x5k.1Licensing provisions : GNU General Public License 3 (GPL)Programming language : C++, HLSLNature of problem : Boussinesq-type models provide the research-level accuracy needed for modeling wave propagation in coastal zones. However the current models, both commercial and open source, do not provide means for real-time computation, nor provide model interactivity and concurrent visualization. In order to achieve a real-time simulation speed in current parallelized models, dozens to hundreds of CPU cores are needed. Celeris is an interactive software which provides faster than real-time simulation and visualization speed on an average user laptop. The novelty of this software is its interactive environment, which allows the user to modify the model and field parameters as the model is running, and to see the effect of these changes immediately.Solution method : A hybrid finite volume-finite difference scheme is used to solve the extended Boussinesq equations. The solver is parallelized using shader programming with Direct3D libraries. Visualization is also performed with the same libraries.

Development of a GIS-based integrated framework for coastal seiches monitoring and forecasting: A North Jiangsu shoal case study

Coastal seiches have become an increasingly important issue in coastal science and present many challenges, particularly when attempting to provide warning services. This paper presents the methodologies, techniques and integrated services adopted for the design and implementation of a Seiches Monitoring and Forecasting Integration Framework (SMAF-IF). The SMAF-IF is an integrated system with different types of sensors and numerical models and incorporates the Geographic Information System (GIS) and web techniques, which focuses on coastal seiche events detection and early warning in the North Jiangsu shoal, China. The in situ sensors perform automatic and continuous monitoring of the marine environment status and the numerical models provide the meteorological and physical oceanographic parameter estimates. A model outputs processing software was developed in C# language using ArcGIS Engine functions, which provides the capabilities of automatically generating visualization maps and warning information. Leveraging the ArcGIS Flex API and ASP.NET web services, a web based GIS framework was designed to facilitate quasi real-time data access, interactive visualization and analysis, and provision of early warning services for end users. The integrated framework proposed in this study enables decision-makers and the publics to quickly response to emergency coastal seiche events and allows an easy adaptation to other regional and scientific domains related to real-time monitoring and forecasting.

PIVE: Per-Iteration Visualization Environment for Real-Time Interactions with Dimension Reduction and Clustering

One of the key advantages of visual analytics is its capability to leverage both humans's visual perception and the power of computing. A big obstacle in integrating machine learning with visual analytics is its high computing cost. To tackle this problem, this paper presents PIVE (Per-Iteration Visualization Environment) that supports real-time interactive visualization with machine learning. By immediately visualizing the intermediate results from algorithm iterations, PIVE enables users to quickly grasp insights and interact with the intermediate output, which then affects subsequent algorithm iterations. In addition, we propose a widely-applicable interaction methodology that allows efficient incorporation of user feedback into virtually any iterative computational method without introducing additional computational cost. We demonstrate the application of PIVE for various dimension reduction algorithms such as multidimensional scaling and t-SNE and clustering and topic modeling algorithms such as k-means and latent Dirichlet allocation.

Temporal Branching Approach for Visual Exploration of Simulation Process in Dynamic Networks

This paper presents a concept for visualization of simulation processes in temporal networks. Core principles are based on interactive real-time visualization of complex networks and dynamic processes. Any modifications in simulation parameters result in division of a timeline into branches. Described concept was integrated into an extended tool for visual analysis and tested on a model of interbank interaction. Proposed developments significantly improve process of visual analysis: pattern detection, search of distinctions and etc.

The Framework for Rapid Graphics Application Development: The Multi-scale Problem Visualization

Abstract Interactive real-time visualization plays a significant role in simulation research domain. Multi-scale problems are in need of high performance visualization with good quality and the same could be said about other problem domains, e.g. big data analysis, physics simulation, etc. The state of the art shows that a universal tool for solving such problem is non-existent. Modern computer graphics requires enormous efforts to implement efficient algorithms on modern GPUs and GAPIs. In the first part of our paper we introduce a framework for rapid graphics application development and its extensions for multi-scale problem visualization. In the second part of the paper we provide a prototype of multi-scale problem's solution in simulation and monitoring of high-precision agent movements starting from behavioral patterns in an airport and up to world-wide flight traffic. Finally we summarize our results and speculate about future investigations.