Visual Analytics Applications Research Articles

Exploring Data Input Problems in Mixed Reality Environments: Proposal and Evaluation of Natural Interaction Techniques

Data input within mixed reality environments poses significant interaction challenges, notably in immersive visual analytics applications. This study assesses five numerical input techniques: three benchmark methods (Touch-Slider, Keyboard, Pinch-Slider) and two innovative multimodal techniques (Bimanual Scaling, Gesture and Voice). An experimental design was employed to compare these techniques’ input efficiency, accuracy, and user experience across varying precision and distance conditions. The findings reveal that multimodal techniques surpass slider methods in input efficiency yet are comparable to keyboards; the voice method excels in reducing cognitive load but falls short in accuracy; and the scaling method marginally leads in user satisfaction but imposes a higher physical load. Furthermore, this study outlines these techniques’ pros and cons and offers design guidelines and future research directions.

Toward Addressing Ambiguous Interactions and Inferring User Intent with Dimension Reduction and Clustering Combinations in Visual Analytics

Direct manipulation interactions on projections are often incorporated in visual analytics applications. These interactions enable analysts to provide incremental feedback to the system in a semi-supervised manner, demonstrating relationships that the analyst wishes to find within the data. However, determining the precise intent of the analyst is a challenge. When an analyst interacts with a projection, the inherent ambiguity of interactions can lead to a variety of possible interpretations that the system can infer. Previous work has demonstrated the utility of clusters as an interaction target to address this “With Respect to What” problem in dimension-reduced projections. However, the introduction of clusters introduces interaction inference challenges as well. In this work, we discuss the interaction space for the simultaneous use of semi-supervised dimension reduction and clustering algorithms. We introduce a novel pipeline representation to disambiguate between interactions on observations and clusters, as well as which underlying model is responding to those analyst interactions. We use a prototype visual analytics tool to demonstrate the effects of these ambiguous interactions, their properties, and the insights that an analyst can glean from each.

HMFVC: A Human-Machine Friendly Video Compression Scheme

Recent studies on Video Coding for Machine (VCM) have achieved remarkable results. However, in practical visual-analytic applications, except for transmitting the visual feature for machine analysis, video textures are also mandatory for human monitoring and decision-making. To this end, this paper proposes a human-machine friendly video compression scheme (HMFVC) which can satisfy both human viewing and machine analysis well. First, we propose a learned semantic representation (LSR) method to extract semantic information between temporal neighboring frames. LSR could be utilized in signal reconstruction for human viewing and visual analysis for machine understanding. Second, given the proposed LSR, we design an end-to-end optimized video compression framework to jointly optimize the visual quality for human perception, analysis accuracy for machines, and compression efficiency as well. Finally, an HMFVC codec is developed, which can achieve higher action recognition accuracy and better reconstruction quality than the traditional codecs and learned video compression approaches. Specifically, HMFVC saves 77% bitrate to achieve the same analysis performance with the original videos compared to x265. To our knowledge, HMFVC is the first end-to-end optimized video compression scheme to serve both humans and machines. It is a promising framework for human-machine friendly video compression approaches.

Context-aware Optimization for Bandwidth-Efficient Image Analytics Offloading

Convolutional Neural Networks (CNN) have given rise to numerous visual analytics applications at the edge of the Internet. The image is typically captured by cameras and then live-streamed to edge servers for analytics due to the prohibitive cost of running CNN on computation-constrained end devices. A critical component to ensure low-latency and accurate visual analytics offloading over low bandwidth networks is image compression which minimizes the amount of visual data to offload and maximizes the decoding quality of salient pixels for analytics. Despite the wide adoption, JPEG standards and traditional image compression techniques do not address the accuracy of analytics tasks, leading to ineffective compression for visual analytics offloading. Although recent machine-centric image compression techniques leverage sophisticated neural network models or hardware architecture to support the accuracy-bandwidth trade-off, they introduce excessive latency in the visual analytics offloading pipeline. This paper presents CICO, a Context-aware Image Compression Optimization framework to achieve low-bandwidth and low-latency visual analytics offloading. CICO contextualizes image compression for offloading by employing easily-computable low-level image features to understand the importance of different image regions for a visual analytics task. Accordingly, CICO can optimize the trade-off between compression size and analytics accuracy. Extensive real-world experiments demonstrate that CICO reduces the bandwidth consumption of existing compression methods by up to 40% under comparable analytics accuracy. Regarding the low-latency support, CICO achieves up to a 2x speedup over state-of-the-art compression techniques.

Vegetation Land Segmentation with Multi-Modal and Multi-Temporal Remote Sensing Images: A Temporal Learning Approach and a New Dataset

In recent years, remote sensing analysis has gained significant attention in visual analysis applications, particularly in segmenting and recognizing remote sensing images. However, the existing research has predominantly focused on single-period RGB image analysis, thus overlooking the complexities of remote sensing image capture, especially in highly vegetated land parcels. In this paper, we provide a large-scale vegetation remote sensing (VRS) dataset and introduce the VRS-Seg task for multi-modal and multi-temporal vegetation segmentation. The VRS dataset incorporates diverse modalities and temporal variations, and its annotations are organized using the Vegetation Knowledge Graph (VKG), thereby providing detailed object attribute information. To address the VRS-Seg task, we introduce VRSFormer, a critical pipeline that integrates multi-temporal and multi-modal data fusion, geometric contour refinement, and category-level classification inference. The experimental results demonstrate the effectiveness and generalization capability of our approach. The availability of VRS and the VRS-Seg task paves the way for further research in multi-modal and multi-temporal vegetation segmentation in remote sensing imagery.

Grand Challenges in Visual Analytics Applications.

In the past two decades, research in visual analytics (VA) applications has made tremendous progress, not just in terms of scientific contributions, but also in real-world impact across wide-ranging domains including bioinformatics, urban analytics, and explainable AI. Despite these success stories, questions on the rigor and value of VA application research have emerged as a grand challenge. This article outlines a research and development agenda for making VA application research more rigorous and impactful. We first analyze the characteristics of VA application research and explain how they cause the rigor and value problem. Next, we propose a research ecosystem for improving scientific value, and rigor and outline an agenda with 12 open challenges spanning four areas, including foundation, methodology, application, and community. We encourage discussions, debates, and innovative efforts toward more rigorous and impactful VA research.

Development and application of visual analysis teaching software for acupoint compatibility laws

A user-friendly teaching software for visual analysis of acupoint compatibility laws has been developed based on the principles of partial order mathematics. This software is designed to provide auxiliary teaching of structured organization and visualization of law knowledge of compatibility data of acupuncture and moxibustion prescriptions from ancient texts, textbooks, and clinical case records. The software is installed as a plugin in the Microsoft Office Excel, allowing the generation of visually appealing graphs and associated rules that align with the cognitive patterns of teachers and students majoring in acupuncture and moxibustion. Its aim is to facilitate the discovery and analysis of underlying patterns and structured knowledge embedded in acupoint compatibility data, thus contributing to the enhancement of teaching effectiveness in acupoint compatibility.

A Rate-Distortion-Classification approach for lossy image compression

In lossy image compression, the objective is to achieve minimal signal distortion while compressing images to a specified bit rate. The increasing demand for visual analysis applications, particularly in classification tasks, has emphasized the significance of considering semantic distortion in compressed images. To bridge the gap between image compression and visual analysis, we propose a Rate-Distortion-Classification (RDC) model for lossy image compression, offering a unified framework to optimize the trade-off between rate, distortion, and classification accuracy. The RDC model is extensively analyzed both statistically on a multi-distribution source and experimentally on the widely used MNIST dataset. The findings reveal that the RDC model exhibits desirable properties, including monotonic non-increasing and convex functions, under certain conditions. This work provides insights into the development of human-machine friendly compression methods and Video Coding for Machine (VCM) approaches, paving the way for end-to-end image compression techniques in real-world applications.

Determining a Trustworthy Application for Medical Data Visualizations through a Knowledge-Based Fuzzy Expert System.

Medical data, such as electronic health records, are a repository for a patient's medical records for use in the diagnosis of different diseases. Using medical data for individual patient care raises a number of concerns, including trustworthiness in data management, privacy, and patient data security. The introduction of visual analytics, a computing system that integrates analytics approaches with interactive visualizations, can potentially deal with information overload concerns in medical data. The practice of assessing the trustworthiness of visual analytics tools or applications using factors that affect medical data analysis is known as trustworthiness evaluation for medical data. It has a variety of major issues, such as a lack of important evaluation of medical data, the need to process much of medical data for diagnosis, the need to make trustworthy relationships clear, and the expectation that it will be automated. Decision-making strategies have been utilized in this evaluation process to avoid these concerns and intelligently and automatically analyze the trustworthiness of the visual analytics tool. The literature study found no hybrid decision support system for visual analytics tool trustworthiness in medical data diagnosis. Thus, this research develops a hybrid decision support system to assess and improve the trustworthiness of medical data for visual analytics tools using fuzzy decision systems. This study examined the trustworthiness of decision systems using visual analytics tools for medical data for the diagnosis of diseases. The hybrid multi-criteria decision-making-based decision support model, based on the analytic hierarchy process and sorting preferences by similarity to ideal solutions in a fuzzy environment, was employed in this study. The results were compared to highly correlated accuracy tests. In conclusion, we highlight the benefits of our proposed study, which includes performing a comparison analysis on the recommended models and some existing models in order to demonstrate the applicability of an optimal decision in real-world environments. In addition, we present a graphical interpretation of the proposed endeavor in order to demonstrate the coherence and effectiveness of our methodology. This research will also help medical experts select, evaluate, and rank the best visual analytics tools for medical data.

WITHDRAWN: RAMPVIS: A visualization and visual analytics infrastructure for COVID-19 data

The COVID-19 pandemic generated large amounts of diverse data, including testing, treatments, vaccine trials, data from modeling, etc. To support epidemiologists and modeling scientists in their efforts to understand and respond to the pandemic, there arose a need for web visualization and visual analytics (VIS) applications to provide insights and support decision-making. In this paper, we present RAMPVIS, an infrastructure designed to support a range of observational, analytical, model-developmental, and dissemination tasks. One of the main features of the system is the ability to "propagate" a visualization designed for one data source to similar ones, this allows a user to quickly visualize large amounts of data. In addition to the COVID pandemic, the RAMPVIS software may be adapted and used with different data to provide rapid visualization support for other emergency responses.

The construction of interactive and multimodal reading in school—a performative, collaborative and dynamic reading

This study aims to demonstrate how interactions between a Visual Analytics (VA) application and students shape an interactive and multimodal reading practice. VA is a technology offering support with analysing vast amounts of data through visualisations. Such information-rich interactive interfaces provide possibilities for students to gain insights, find correlations, draw conclusions and gain insights, but they also generate complexities concerning how to ‘read’ multimodal information on a screen. Inspired by Design-Based Research, interventions were designed and conducted in five social science secondary classrooms. The interactions between the VA application Statistics eXplorer and the students were video captured. A socio-material semiotic approach guides the analyses of how interactions between all actors (the interactive visualisations, the written text, the teachers, students, etc.) produce a reading network. The results show a reading characterised by being performative, collaborative, and dynamic. A combination of visuals and text supports the reading. However, visuals such as colour, highlighting and movement dominantly attract students’ attention, while written text often becomes subordinate and sometimes even ‘invisible’. Hence, this paper argues that it is vital for teachers to didactically support students’ visual reading skills.

A Survey on Progressive Visualization.

Currently, growing data sources and long-running algorithms impede user attention and interaction with visual analytics applications. Progressive visualization (PV) and visual analytics (PVA) alleviate this problem by allowing immediate feedback and interaction with large datasets and complex computations, avoiding waiting for complete results by using partial results improving with time. Yet, creating a progressive visualization requires more effort than a regular visualization but also opens up new possibilities, such as steering the computations towards more relevant parts of the data, thus saving computational resources. However, there is currently no comprehensive overview of the design space for progressive visualization systems. We surveyed the related work of PV and derived a new taxonomy for progressive visualizations by systematically categorizing all PV publications that included visualizations with progressive features. Progressive visualizations can be categorized by well-known visualization taxonomies, but we also found that progressive visualizations can be distinguished by the way they manage their data processing, data domain, and visual update. Furthermore, we identified key properties such as uncertainty, steering, visual stability, and real-time processing that are significantly different with progressive applications. We also collected evaluation methodologies reported by the publications and conclude with statistical findings, research gaps, and open challenges. A continuously updated visual browser of the survey data is available at visualsurvey.net/pva.

Sequoia: A Framework for Visual Analysis of RNA Modifications from Direct RNA Sequencing Data.

Oxford Nanopore-based long-read direct RNA sequencing protocols are being increasingly used to study the dynamics of RNA metabolic processes due to improvements in read lengths, increased throughput, decreasing cost, ease of library preparation, and convenience. Long-read sequencing enables single-molecule-based detection of posttranscriptional changes, promising novel insights into the functional roles of RNA. However, fulfilling this potential will necessitate the development of new tools for analyzing and exploring this type of data. Although there are tools that allow users to analyze signal information, such as comparing raw signal traces to a nucleotide sequence, they don't facilitate studying each individual signal instance in each read or perform analysis of signal clusters based on signal similarity. Therefore, we present Sequoia, a visual analytics application that allows users to interactively analyze signals originating from nanopore sequencers and can readily be extended to both RNA and DNA sequencing datasets. Sequoia combines a Python-based backend with a multi-view graphical interface that allows users to ingest raw nanopore sequencing data in Fast5 format, cluster sequences based on electric-current similarities, and drill-down onto signals to find attributes of interest. In this tutorial, we illustrate each individual step involved in running Sequoia and in the process dissect input data characteristics. We show how to generate Nanopore sequencing-based visualizations by leveraging dimensionality reduction and parameter tuning to separate modified RNA sequences from their unmodified counterparts. Sequoia's interactive features enhance nanopore-based computational methodologies. Sequoia enables users to construct rationales and hypotheses and develop insights about the dynamic nature of RNA from the visual analysis. Sequoia is available at https://github.com/dnonatar/Sequoia .

Intelligent visualization and visual analytics

Intelligent visualization and visual analytics

Ultra-High Temporal Resolution Visual Reconstruction From a Fovea-Like Spike Camera via Spiking Neuron Model.

Neuromorphic vision sensor is a new bio-inspired imaging paradigm emerged in recent years. It uses the asynchronous spike signals instead of the traditional frame-based manner to achieve ultra-high speed sampling. Unlike the dynamic vision sensor (DVS) that perceives movement by imitating the retinal periphery, the spike camera was developed recently to perceive fine textures by simulating a small retinal region called the fovea. For this new type of neuromorphic camera, how to reconstruct ultra-high speed visual images from spike data becomes an important yet challenging issue in visual scene perception, analysis, and recognition applications. In this paper, a bio-inspired visual reconstruction framework for the spike camera is proposed for the first time. Its core idea is to use the biologically inspired adaptive adjustment mechanisms, combined with the spatiotemporal spike information extracted by the proposed model, to reconstruct the full texture of natural scenes in an ultra-high temporal resolution. Specifically, the proposed model consists of a motion local excitation layer, a spike refining layer and a visual reconstruction layer motivated by the bio-realistic leaky integrate-and-fire (LIF) neurons and synapse connection with spike-timing dependent plasticity (STDP) rule. To evaluate the performance, a spike dataset was constructed for normal and high-speed scenes in real-world recorded by the spike camera. The experimental results show that the proposed approach can reconstruct the visual images with 40,000 frames per second in both normal and high-speed scenes, while achieving high dynamic range and high image quality.

SkiVis: Visual Exploration and Route Planning in Ski Resorts.

Optimal ski route selection is a challenge based on a multitude of factors, such as the steepness, compass direction, or crowdedness. The personal preferences of every skier towards these factors require individual adaptations, which aggravate this task. Current approaches within this domain do not combine automated routing capabilities with user preferences, missing out on the possibility of integrating domain knowledge in the analysis process. We introduce SkiVis, a visual analytics application to interactively explore ski slopes and provide routing recommendations based on user preferences. In collaboration with ski guides and enthusiasts, we elicited requirements and guidelines for such an application and propose different workflows depending on the skiers' familiarity with the resort. In a case study on the resort of Ski Arlberg, we illustrate how to leverage volunteered geographic information to enable a numerical comparison between slopes. We evaluated our approach through a pair-analytics study and demonstrate how it supports skiers in discovering relevant and preference-based ski routes. Besides the tasks investigated in the study, we derive additional use cases from the interviews that showcase the further potential of SkiVis, and contribute directions for further research opportunities.

A flexible framework for visualizing and exploring patient misdiagnosis over time

Diagnosis is a complex and ambiguous process and yet, it is the critical hinge point for all subsequent clinical reasoning and decision-making. Tracking the quality of the patient diagnostic process has the potential to provide valuable insights in improving the diagnostic accuracy and to reduce downstream errors but needs to be informative, timely, and efficient at scale. However, due to the rate at which healthcare data are captured on a daily basis, manually reviewing the diagnostic history of each patient would be a severely taxing process without efficient data reduction and representation. Application of data visualization and visual analytics to healthcare data is one promising approach for addressing these challenges. This paper presents a novel flexible visualization and analysis framework for exploring the patient diagnostic process over time (i.e., patient diagnosis paths). Our framework allows users to select a specific set of patients, events and/or conditions, filter data based on different attributes, and view further details on the selected patient cohort while providing an interactive view of the resulting patient diagnosis paths. A practical demonstration of our system is presented with a case study exploring infection-based patient diagnosis paths.

FluxomicsExplorer: Differential visual analysis of Flux Sampling based on Metabolomics

Flux Sampling (FS) is an unbiased constraint-based modeling method in systems biology that characterizes the admissible metabolic flux space of cells and organisms at a molecular level. It returns information about reaction flux ranges and their probability from a genome-scale metabolic model (GEM), which can be used to predict metabolic phenotypes. FS is particularly beneficial if a cellular objective cannot be easily identified. We present FluxomicsExplorer, a web-based visual analysis application that facilitates a data- and task-driven exploratory analysis of FS results. It was designed together with experts in medicine and systems biology to satisfy their requirements and offers multiple linked views that visualize the standardized scoring of flux differences, flux distributions, and the network of biological pathways to guide the analysis process. Instead of focusing on a subset of known influential reactions for the analysis, it allows users to assess the results as a whole concerning the underlying GEM. The visualizations guide the user to affected reactions, metabolites, and network areas by highlighting relevant changes between two conditions. These flux changes can then be analyzed in detail by visualizing the reaction flux distributions. We show FluxomicsExplorer’s utility using a case study of the red blood cell metabolism in different blood plasma environments.

SMART CAMPUS ELECTRICITY DATA VISUAL ANALYSIS SYSTEM

Abstract. Smart grid is the basic support of smart city development. The application of visual analytics to electricity system helps monitor and analyse the characteristic information in electricity data, which provides a strong guarantee for mastering the operation status of electricity system and achieving effective energy planning. However, as the complexity and size of electricity data continues to grow, it increases the burden on electricity workers to understand and analyse the electricity consumption situation. In response to these problems, a new type of electricity data visualisation and analysis system has been proposed, which enables interactive analysis of large amounts of electricity data. The system has the following advantages. First, A novel visualisation graphic has been designed and implemented to enable electricity workers to visualise a comprehensive picture of electricity consumption at different granularities. Second, designing appropriate visualisations to highlight the characteristics of the data itself, depending on the specific needs and type of data. Finally, the system provides a set of coupled visual views and interactions to support system users to freely explore the campus electricity situation from multiple scales.

Mixed Reality Based Visual Analytics

Having a novel human-computer interaction mode, mixed reality expands the design space of visualization and interactive interface, confirming itself a broad application prospect. Recently, mixed reality technologies have undergone rapid developments, and many researchers have applied it into the field of visual analytics. In this situation, first the concept and current developments of mixed reality are introduced, the differences between the mixed reality environment and the traditional interactive environment from aspects of display and interaction are discussed, the advantages and limitations of mixed reality visual analytics are analyzed, and related research projects are presented. In addition, based on the three application scenarios of mixed reality visual analytics, the augmenting visualization, the situated analytics, and the collaborative analytics, current works of the mixed reality visual analysis application are also introduced. Finally, current progress are summarized and the opportunities and challenges of mixed reality visual analytics are discussed.