Visual Analytics Solution Research Articles

Enhancing Visual Analytics systems with guidance: A task-driven methodology

Enhancing Visual Analytics (VA) systems with guidance, such as the automated provision of data-driven suggestions and answers to the user’s task, is becoming increasingly important and common. However, how to design such systems remains a challenging task. We present a methodology to aid and structure the design of guidance for enhancing VA solutions consisting of four steps: (S1) defining the target of analysis, (S2) identifying the user tasks, (S3) describing the guidance tasks, and (S4) placing guidance. In summary, our proposed methodology specifies a space of possible user tasks and maps them to the corresponding space of guidance tasks, using recent abstract task typologies for guidance and visualization. We exemplify this methodology through two case studies from the literature: Overview, a system for exploring and labeling document collections aimed at journalists, and DoRIAH, a system for historical imagery analysis. We show how our methodology enriches existing VA solutions with guidance and provides a structured way to design guidance in complex VA scenarios.

Interactive Visual Analysis of Spatial Sensitivities.

Sensitivity analyses of simulation ensembles determine how simulation parameters influence the simulation's outcome. Commonly, one global numerical sensitivity value is computed per simulation parameter. However, when considering 3D spatial simulations, the analysis of localized sensitivities in different spatial regions is of importance in many applications. For analyzing the spatial variation of parameter sensitivity, one needs to compute a spatial sensitivity scalar field per simulation parameter. Given n simulation parameters, we obtain multi-field data consisting of n scalar fields when considering all simulation parameters. We propose an interactive visual analytics solution to analyze the multi-field sensitivity data. It supports the investigation of how strongly and in what way individual parameters influence the simulation outcome, in which spatial regions this is happening, and what the interplay of the simulation parameters is. Its central component is an overview visualization of all sensitivity fields that avoids 3D occlusions by linearizing the data using an adapted scheme of data-driven space-filling curves. The spatial sensitivity values are visualized in a combination of a Horizon Graph and a line chart. We validate our approach by applying it to synthetic and real-world ensemble data.

Visual analysis of model parameter sensitivities along warm conveyor belt trajectories using Met.3D (1.6.0-multivar1)

Abstract. Numerical weather prediction models rely on parameterizations for subgrid-scale processes, e.g., for cloud microphysics, which are a well-known source of uncertainty in weather forecasts. Via algorithmic differentiation, which computes the sensitivities of prognostic variables to changes in model parameters, these uncertainties can be quantified. In this article, we present visual analytics solutions to analyze interactively the sensitivities of a selected prognostic variable to multiple model parameters along strongly ascending trajectories, so-called warm conveyor belt (WCB) trajectories. We propose a visual interface that enables us to (a) compare the values of multiple sensitivities at a single time step on multiple trajectories, (b) assess the spatiotemporal relationships between sensitivities and the trajectories' shapes and locations, and (c) find similarities in the temporal development of sensitivities along multiple trajectories. We demonstrate how our approach enables atmospheric scientists to interactively analyze the uncertainty in the microphysical parameterizations and along the trajectories with respect to the selected prognostic variable. We apply our approach to the analysis of WCB trajectories within extratropical Cyclone Vladiana, which occurred between 22–25 September 2016 over the North Atlantic. Peaks of sensitivities that occur at different times relative to a trajectory's fastest ascent reveal that trajectories with their fastest ascent in the north are more susceptible to rain sedimentation from above than trajectories that ascend further south. In contrast, large sensitivities to cloud condensation nuclei (CCN) activation and cloud droplet collision in the south indicate a local rain droplet formation. These large sensitivities reveal considerable uncertainty in the shape of clouds and subsequent rainfall. Sensitivities to cloud droplets' formation and subsequent conversion to rain droplets are also more pronounced along convective ascending trajectories than for slantwise ascents. The slantwise ascending trajectories are characterized by periods of slower ascent and even descent, during which the sensitivities to the formation of cloud droplets and rain droplets alternate. This alternating pattern leads to large-scale precipitation patterns, whereas convective ascending trajectories do not exhibit this pattern. Thus the primary source for uncertainty in large-scale precipitation patterns stems from slantwise ascents. The strong ascent of convective trajectories leads to large sensitivities of rain mass density to riming and freezing parameters at high altitudes, which are barely present in slantwise ascending trajectories. These sensitivities correspond to uncertainties concerning graupel and hail formation in convective ascents.

Fleet Profile: Using visual analytics to prospect logistic solutions in industrial vehicles fleet

Fleet planning and management activities are essential to establish the quantity and type of vehicles needed to pursue production plans and reduce costs. In steel companies, logistic analysts are responsible for these fleet activities. However, analysts still face challenges assessing fleet utilization due to the volume and form in which the data is found. Normally, this type of problem is out of the scope of fleet planning models in the literature. For information extraction from massive and complex databases, visual analytics is an alternative that employs data analysis and visualization techniques. This research investigates visual analytics to support industrial vehicle fleet management. As a result, two artifacts were developed: a fleet measurement model and Fleet Profile, a visual analytics solution. Two research cycles, each one producing a Fleet Profile version that was evaluated with real cases after the development. Findings from the analysis of the evaluation contents indicated that the Fleet Profile’s visual description could help to evaluate fleet utilization and identify gaps for fleet optimization.

State of the Art of Visual Analytics for eXplainable Deep Learning

AbstractThe use and creation of machine‐learning‐based solutions to solve problems or reduce their computational costs are becoming increasingly widespread in many domains. Deep Learning plays a large part in this growth. However, it has drawbacks such as a lack of explainability and behaving as a black‐box model. During the last few years, Visual Analytics has provided several proposals to cope with these drawbacks, supporting the emerging eXplainable Deep Learning field. This survey aims to (i) systematically report the contributions of Visual Analytics for eXplainable Deep Learning; (ii) spot gaps and challenges; (iii) serve as an anthology of visual analytical solutions ready to be exploited and put into operation by the Deep Learning community (architects, trainers and end users) and (iv) prove the degree of maturity, ease of integration and results for specific domains. The survey concludes by identifying future research challenges and bridging activities that are helpful to strengthen the role of Visual Analytics as effective support for eXplainable Deep Learning and to foster the adoption of Visual Analytics solutions in the eXplainable Deep Learning community. An interactive explorable version of this survey is available online at https://aware‐diag‐sapienza.github.io/VA4XDL.

Visual Exploration of Financial Data with Incremental Domain Knowledge.

Modelling the dynamics of a growing financial environment is a complex task that requires domain knowledge, expertise and access to heterogeneous information types. Such information can stem from several sources at different scales, complicating the task of forming a holistic impression of the financial landscape, especially in terms of the economical relationships between firms. Bringing this scattered information into a common context is, therefore, an essential step in the process of obtaining meaningful insights about the state of an economy. In this paper, we present Sabrina2.0, a Visual Analytics (VA) approach for exploring financial data across different scales, from individual firms up to nation-wide aggregate data. Our solution is coupled with a pipeline for the generation of firm-to-firm financial transaction networks, fusing information about individual firms with sector-to-sector transaction data and domain knowledge on macroscopic aspects of the economy. Each network can be created to have multiple instances to compare different scenarios. We collaborated with experts from finance and economy during the development of our VA solution, and evaluated our approach with seven domain experts across industry and academia through a qualitative insight-based evaluation. The analysis shows how Sabrina2.0 enables the generation of insights, and how the incorporation of transaction models assists users in their exploration of a national economy.

Beyond Click-and-View: a Comparative Study of Data Management Approaches for Interactive Visualization

Visual analytics solutions have been growing in popularity in recent years, not only for showing final results but also for assisting in interactive analysis and decision-making. Analysis of a large amount of data requires flexible exploration and visualizations. However, queries that span geographical regions over time slices are expensive to compute, which turns it challenging to accomplish interactive speeds for huge data sets. Such systems require efficient data availability, so that response time does not interfere with the user’s ability to observe and analyze. Simultaneously, researches in the database domain have proposed solutions that can be used to support visualization systems. This article presents a comparative study of data management approaches to support interactive visualizations. The chosen data management solutions are (i) Apache Drill (a Polystore system), (ii) Apache Spark (a big data framework), (iii) Elasticsearch (a search engine), (iv) MonetDB (a column-oriented DBMS), and (v) PostgreSQL (a relational DBMS). To evaluate the performance of each solution, we selected a list of spatiotemporal queries among multiple queries submitted by users in a visual analytics system for rainfall data analysis named TEMPO. The results of this study show that Apache Spark and MonetDB present the best performance for the selected queries.

ModelWise: Interactive Model Comparison for Model Diagnosis, Improvement and Selection

AbstractModel comparison is an important process to facilitate model diagnosis, improvement, and selection when multiple models are developed for a classification task. It involves careful comparison concerning model performance and interpretation. Current visual analytics solutions often ignore the feature selection process. They either do not support detailed analysis of multiple multi‐class classifiers or rely on feature analysis alone to interpret model results. Understanding how different models make classification decisions, especially classification disagreements of the same instances, requires a deeper model understanding. We present ModelWise, a visual analytics method to compare multiple multi‐class classifiers in terms of model performance, feature space, and model explanation. ModelWise adapts visualizations with rich interactions to support multiple workflows to achieve model diagnosis, improvement, and selection. It considers feature subspaces generated for use in different models and improves model understanding by model explanation. We demonstrate the usability of ModelWise with two case studies, one with a small exemplar dataset and another developed with a machine learning expert with real‐world perioperative data.

Interactivized: Visual Interaction for Better Decisions With Interactive Multiobjective Optimization

In today’s data-driven world, decision makers are facing many conflicting objectives. Since there is usually no solution that optimizes all objectives simultaneously, the aim is to identify a solution with acceptable trade-offs. Interactive multiobjective optimization methods are iterative processes in which a human decision maker repeatedly provides one’s preferences to request computing new solutions and compares them. With these methods, the decision maker can learn about the problem and its limitations. However, advanced optimization software usually offer simple visualization tools that can be significantly improved. On the other hand, current approaches for multiobjective optimization from the visualization community provide superior visualization tools but lack advanced optimization. In this paper, we introduce a new term, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interactivize</i> , for integrating interactive multiobjective optimization and interactive visualization and present an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interactivized</i> approach supporting decision makers in visually steering interactive multiobjective optimization methods. We integrate state-of-the-art interactive visualization with the process of interactive multiobjective optimization in a visual analytics solution that significantly improves the analysis workflow of decision makers, like comparing selected solutions and specifying new preferences during the iterative solution process. To realize the new interactivized approach, we combine a coordinated multiple views system with DESDEO, an open-source software framework for interactive multiobjective optimization. We demonstrate our interactivized approach on a river pollution problem.

USEVis: Visual analytics of attention-based neural embedding in information retrieval

Neural attention-based encoders, which effectively attend sentence tokens to their associated context without being restricted by long-term distance or dependency, have demonstrated outstanding performance in embedding sentences into meaningful representations (embeddings). The Universal Sentence Encoder (USE) is one of the most well-recognized deep neural network (DNN) based solutions, which is facilitated with an attention-driven transformer architecture and has been pre-trained on a large number of sentences from the Internet. Besides the fact that USE has been widely used in many downstream applications, including information retrieval (IR), interpreting its complicated internal working mechanism remains challenging. In this work, we present a visual analytics solution towards addressing this challenge. Specifically, focused on semantics and syntactics (concepts and relations) that are critical to domain clinical IR, we designed and developed a visual analytics system, i.e., USEVis. The system investigates the power of USE in effectively extracting sentences’ semantics and syntactics through exploring and interpreting how linguistic properties are captured by attentions. Furthermore, by thoroughly examining and comparing the inherent patterns of these attentions, we are able to exploit attentions to retrieve sentences/documents that have similar semantics or are closely related to a given clinical problem in IR. By collaborating with domain experts, we demonstrate use cases with inspiring findings to validate the contribution of our work and the effectiveness of our system.

P6: A Declarative Language for Integrating Machine Learning in Visual Analytics.

We present P6, a declarative language for building high performance visual analytics systems through its support for specifying and integrating machine learning and interactive visualization methods. As data analysis methods based on machine learning and artificial intelligence continue to advance, a visual analytics solution can leverage these methods for better exploiting large and complex data. However, integrating machine learning methods with interactive visual analysis is challenging. Existing declarative programming libraries and toolkits for visualization lack support for coupling machine learning methods. By providing a declarative language for visual analytics, P6 can empower more developers to create visual analytics applications that combine machine learning and visualization methods for data analysis and problem solving. Through a variety of example applications, we demonstrate P6's capabilities and show the benefits of using declarative specifications to build visual analytics systems. We also identify and discuss the research opportunities and challenges for declarative visual analytics.

Revisiting the Modifiable Areal Unit Problem in Deep Traffic Prediction with Visual Analytics.

Deep learning methods are being increasingly used for urban traffic prediction where spatiotemporal traffic data is aggregated into sequentially organized matrices that are then fed into convolution-based residual neural networks. However, the widely known modifiable areal unit problem within such aggregation processes can lead to perturbations in the network inputs. This issue can significantly destabilize the feature embeddings and the predictions - rendering deep networks much less useful for the experts. This paper approaches this challenge by leveraging unit visualization techniques that enable the investigation of many-to-many relationships between dynamically varied multi-scalar aggregations of urban traffic data and neural network predictions. Through regular exchanges with a domain expert, we design and develop a visual analytics solution that integrates 1) a Bivariate Map equipped with an advanced bivariate colormap to simultaneously depict input traffic and prediction errors across space, 2) a Moran's I Scatterplot that provides local indicators of spatial association analysis, and 3) a Multi-scale Attribution View that arranges non-linear dot plots in a tree layout to promote model analysis and comparison across scales. We evaluate our approach through a series of case studies involving a real-world dataset of Shenzhen taxi trips, and through interviews with domain experts. We observe that geographical scale variations have important impact on prediction performances, and interactive visual exploration of dynamically varying inputs and outputs benefit experts in the development of deep traffic prediction models.

Towards Better Bus Networks: A Visual Analytics Approach

Bus routes are typically updated every 3-5 years to meet constantly changing travel demands. However, identifying deficient bus routes and finding their optimal replacements remain challenging due to the difficulties in analyzing a complex bus network and the large solution space comprising alternative routes. Most of the automated approaches cannot produce satisfactory results in real-world settings without laborious inspection and evaluation of the candidates. The limitations observed in these approaches motivate us to collaborate with domain experts and propose a visual analytics solution for the performance analysis and incremental planning of bus routes based on an existing bus network. Developing such a solution involves three major challenges, namely, a) the in-depth analysis of complex bus route networks, b) the interactive generation of improved route candidates, and c) the effective evaluation of alternative bus routes. For challenge a, we employ an overview-to-detail approach by dividing the analysis of a complex bus network into three levels to facilitate the efficient identification of deficient routes. For challenge b, we improve a route generation model and interpret the performance of the generation with tailored visualizations. For challenge c, we incorporate a conflict resolution strategy in the progressive decision-making process to assist users in evaluating the alternative routes and finding the most optimal one. The proposed system is evaluated with two usage scenarios based on real-world data and received positive feedback from the experts.

A Survey of Visual Analytics for Public Health

AbstractWe describe visual analytics solutions aiming to support public health professionals, and thus, preventive measures. Prevention aims at advocating behaviour and policy changes likely to improve human health. Public health strives to limit the outbreak of acute diseases as well as the reduction of chronic diseases and injuries. For this purpose, data are collected to identify trends in human health, to derive hypotheses, e.g. related to risk factors, and to get insights in the data and the underlying phenomena. Most public health data have a temporal character. Moreover, the spatial character, e.g. spatial clustering of diseases, needs to be considered for decision‐making. Visual analytics techniques involve (subspace) clustering, interaction techniques to identify relevant subpopulations, e.g. being particularly vulnerable to diseases, imputation of missing values, visual queries as well as visualization and interaction techniques for spatio‐temporal data. We describe requirements, tasks and visual analytics techniques that are widely used in public health before going into detail with respect to applications. These include outbreak surveillance and epidemiology research, e.g. cancer epidemiology. We classify the solutions based on the visual analytics techniques employed. We also discuss gaps in the current state of the art and resulting research opportunities in a research agenda to advance visual analytics support in public health.

Personalized Sketch-Based Brushing in Scatterplots.

Brushing is at the heart of most modern visual analytics solutions and effective and efficient brushing is crucial for successful interactive data exploration and analysis. As the user plays a central role in brushing, several data-driven brushing tools have been designed that are based on predicting the user's brushing goal. All of these general brushing models learn the users' average brushing preference, which is not optimal for every single user. In this paper, we propose an innovative framework that offers the user opportunities to improve the brushing technique while using it. We realized this framework with a CNN-based brushing technique and the result shows that with additional data from a particular user, the model can be refined (better performance in terms of accuracy), eventually converging to a personalized model based on a moderate amount of retraining.

MAD: A visual analytics solution for Multi-step cyber Attacks Detection

Abstract Software vulnerabilities represent one of the main weaknesses of an Information Technology (IT) system w.r.t. cyber attacks and nowadays consolidated official data, like the Common Vulnerability Exposure (CVE) dictionary, provide precise and reliable details about them. This information, together with the identification of priority systems to defend allows for inspecting the network structure and the most probable paths an attacker is likely to follow to reach sensible resources, with the main goal of identify suitable mitigation actions that reduce the risk of an attack. Some of these mitigation actions can be applied without further delay, some of them, instead, imply a high operational impact on the organization business that makes their usage convenient only when an attack is really on the way. Dealing with this issue is particularly challenging in the context of critical infrastructure where, even if patches are available, organization mission constraints create obstacles to their straightforward application. In this scenario, security operators are forced to deal with known vulnerabilities that cannot be patched and they spend a noticeable effort in proactive analysis, devising countermeasures that can mitigate the effect of a possible attack. This paper presents a Multi-step cyber Attack Detection (MAD) Visual Analytics solution aiming at assisting security operators in improving their network security by analyzing the possible attacks and identifying suitable mitigations. Moreover, during an attack, the system visually presents the security operator with the relevant pieces of information allowing a better comprehension of the attack status and its probable evolution, in order to make decisions on the possible countermeasures.

Visualization and analysis of schema and instances of ontologies for improving user tasks and knowledge discovery

Abstract Ontologies are an important resource for knowledge representation. Their structure can be complex due to role relations between several concepts, distinct attributes, and different instances. In this paper, we discuss a Visual Analytics solution, relying on the use of multiple coordinated views for exploring different ontology aspects and a novel use of the degree of interest (DoI) suppression technique to reduce the complexity of the ontology visual representation. Visual Analytics facilitates the understanding of the domain and tasks represented by ontologies, thus allowing to carry out exploratory analysis to optimize the comprehension of data semantics including non-explicit relationships between data. Through the DoI technique, we place the main concept in focus, distinguishing it from the unnecessary information and facilitating the analysis and understanding of correlated data. We evaluated all the devised solutions, and the results reinforce the importance of providing visualization and analysis techniques dedicated to the schema and instances levels of ontologies for the discovery of non-explicit information.

The Validity, Generalizability and Feasibility of Summative Evaluation Methods in Visual Analytics

Many evaluation methods have been used to assess the usefulness of Visual Analytics (VA) solutions. These methods stem from a variety of origins with different assumptions and goals, which cause confusion about their proofing capabilities. Moreover, the lack of discussion about the evaluation processes may limit our potential to develop new evaluation methods specialized for VA. In this paper, we present an analysis of evaluation methods that have been used to summatively evaluate VA solutions. We provide a survey and taxonomy of the evaluation methods that have appeared in the VAST literature in the past two years. We then analyze these methods in terms of validity and generalizability of their findings, as well as the feasibility of using them. We propose a new metric called summative quality to compare evaluation methods according to their ability to prove usefulness, and make recommendations for selecting evaluation methods based on their summative quality in the VA domain.

Vulnus: Visual Vulnerability Analysis for Network Security.

Vulnerabilities represent one of the main weaknesses of IT systems and the availability of consolidated official data, like CVE (Common Vulnerabilities and Exposures), allows for using them to compute the paths an attacker is likely to follow. However, even if patches are available, business constraints or lack of resources create obstacles to their straightforward application. As a consequence, the security manager of a network needs to deal with a large number of vulnerabilities, making decisions on how to cope with them. This paper presents VULNUS (VULNerabilities visUal aSsessment), a visual analytics solution for dynamically inspecting the vulnerabilities spread on networks, allowing for a quick understanding of the network status and visually classifying nodes according to their vulnerabilities. Moreover, VULNUS computes the approximated optimal sequence of patches able to eliminate all the attack paths and allows for exploring sub-optimal patching strategies, simulating the effect of removing one or more vulnerabilities. VULNUS has been evaluated by domain experts using a lab-test experiment, investigating the effectiveness and efficiency of the proposed solution.

RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records.

We have recently seen many successful applications of recurrent neural networks (RNNs) on electronic medical records (EMRs), which contain histories of patients' diagnoses, medications, and other various events, in order to predict the current and future states of patients. Despite the strong performance of RNNs, it is often challenging for users to understand why the model makes a particular prediction. Such black-box nature of RNNs can impede its wide adoption in clinical practice. Furthermore, we have no established methods to interactively leverage users' domain expertise and prior knowledge as inputs for steering the model. Therefore, our design study aims to provide a visual analytics solution to increase interpretability and interactivity of RNNs via a joint effort of medical experts, artificial intelligence scientists, and visual analytics researchers. Following the iterative design process between the experts, we design, implement, and evaluate a visual analytics tool called RetainVis, which couples a newly improved, interpretable, and interactive RNN-based model called RetainEX and visualizations for users' exploration of EMR data in the context of prediction tasks. Our study shows the effective use of RetainVis for gaining insights into how individual medical codes contribute to making risk predictions, using EMRs of patients with heart failure and cataract symptoms. Our study also demonstrates how we made substantial changes to the state-of-the-art RNN model called RETAIN in order to make use of temporal information and increase interactivity. This study will provide a useful guideline for researchers that aim to design an interpretable and interactive visual analytics tool for RNNs.