Interactive Visual Analytics System Research Articles

M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis.

Multimodal sentiment analysis aims to recognize people's attitudes from multiple communication channels such as verbal content (i.e., text), voice, and facial expressions. It has become a vibrant and important research topic in natural language processing. Much research focuses on modeling the complex intra- and inter-modal interactions between different communication channels. However, current multimodal models with strong performance are often deep-learning-based techniques and work like black boxes. It is not clear how models utilize multimodal information for sentiment predictions. Despite recent advances in techniques for enhancing the explainability of machine learning models, they often target unimodal scenarios (e.g., images, sentences), and little research has been done on explaining multimodal models. In this paper, we present an interactive visual analytics system, M2 Lens, to visualize and explain multimodal models for sentiment analysis. M2 Lens provides explanations on intra- and inter-modal interactions at the global, subset, and local levels. Specifically, it summarizes the influence of three typical interaction types (i.e., dominance, complement, and conflict) on the model predictions. Moreover, M2 Lens identifies frequent and influential multimodal features and supports the multi-faceted exploration of model behaviors from language, acoustic, and visual modalities. Through two case studies and expert interviews, we demonstrate our system can help users gain deep insights into the multimodal models for sentiment analysis.

Toward Better Understanding of Dynamic Soccer Passing Networks

<p indent=0mm>Recent studies have confirmed that analysis of passing networks of soccer can provide insights for understanding teams’ attacking strategies. However, most of the existing works regarded passing networks as static networks, failing to investigate the change of strategies and time-varying passing patterns. To address the problem, important tasks from interviews with domain experts are identified and an interactive visual analytics system is developed to support comprehensive analysis of the dynamics of passing networks. A glyph-based design accompanied with projection techniques is proposed to visualize the dynamic change in network topologies, the associated spatial context, and the network characteristics. With the system, users can detect the changing pattern of passes and analyze the relationship between passes and spatial regions. For evaluation, expert interviews are conducted to demonstrate the usability of the system in analyzing the dynamics of passing networks.

Bridging cognitive gaps between user and model in interactive dimension reduction

Interactive machine learning (ML) systems are difficult to design because of the “Two Black Boxes” problem that exists at the interface between human and machine. Many algorithms that are used in interactive ML systems are black boxes that are presented to users, while the human cognition represents a second black box that can be difficult for the algorithm to interpret. These black boxes create cognitive gaps between the user and the interactive ML model. In this paper, we identify several cognitive gaps that exist in a previously-developed interactive visual analytics (VA) system, Andromeda, but are also representative of common problems in other VA systems. Our goal with this work is to open both black boxes and bridge these cognitive gaps by making usability improvements to the original Andromeda system. These include designing new visual features to help people better understand how Andromeda processes and interacts with data, as well as improving the underlying algorithm so that the system can better implement the intent of the user during the data exploration process. We evaluate our designs through both qualitative and quantitative analysis, and the results confirm that the improved Andromeda system outperforms the original version in a series of high-dimensional data analysis tasks.

Interactive Visual Exploration of Longitudinal Historical Career Mobility Data.

The increased availability of quantitative historical datasets has provided new research opportunities for multiple disciplines in social science. In this article, we work closely with the constructors of a new dataset, CGED-Q (China Government Employee Database-Qing), that records the career trajectories of over 340,000 government officials in the Qing bureaucracy in China from 1760 to 1912. We use these data to study career mobility from a historical perspective and understand social mobility and inequality. However, existing statistical approaches are inadequate for analyzing career mobility in this historical dataset with its fine-grained attributes and long time span, since they are mostly hypothesis-driven and require substantial effort. We propose CareerLens, an interactive visual analytics system for assisting experts in exploring, understanding, and reasoning from historical career data. With CareerLens, experts examine mobility patterns in three levels-of-detail, namely, the macro-level providing a summary of overall mobility, the meso-level extracting latent group mobility patterns, and the micro-level revealing social relationships of individuals. We demonstrate the effectiveness and usability of CareerLens through two case studies and receive encouraging feedback from follow-up interviews with domain experts.

Githru: Visual Analytics for Understanding Software Development History Through Git Metadata Analysis.

Git metadata contains rich information for developers to understand the overall context of a large software development project. Thus it can help new developers, managers, and testers understand the history of development without needing to dig into a large pile of unfamiliar source code. However, the current tools for Git visualization are not adequate to analyze and explore the metadata: They focus mainly on improving the usability of Git commands instead of on helping users understand the development history. Furthermore, they do not scale for large and complex Git commit graphs, which can play an important role in understanding the overall development history. In this paper, we present Githru, an interactive visual analytics system that enables developers to effectively understand the context of development history through the interactive exploration of Git metadata. We design an interactive visual encoding idiom to represent a large Git graph in a scalable manner while preserving the topological structures in the Git graph. To enable scalable exploration of a large Git commit graph, we propose novel techniques (graph reconstruction, clustering, and Context-Preserving Squash Merge (CSM) methods) to abstract a large-scale Git commit graph. Based on these Git commit graph abstraction techniques, Githru provides an interactive summary view to help users gain an overview of the development history and a comparison view in which users can compare different clusters of commits. The efficacy of Githru has been demonstrated by case studies with domain experts using real-world, in-house datasets from a large software development team at a major international IT company. A controlled user study with 12 developers comparing Githru to previous tools also confirms the effectiveness of Githru in terms of task completion time.

Visual exploration of urban functional zones based on augmented nonnegative tensor factorization

A city contains a variety of different urban functions with referring to the purpose of land use to support the diverse needs of urban residents, such as residence, working and recreation. Exploring urban functional zones is a critical task, which provides valuable applications for business site selection, transportation management and urban planning. It is well known that location information and human mobility semantics both are significant factors for identifying urban functional zones, just considering one factor is not effective. However, most of existing techniques capture the transformation of functional zones and interpret the results based solely on location or mobility semantics, and lack the capacity to deal with the multifaceted features of urban data. To tackle these problems, in this paper, we propose an interactive visual analytics system for effectively exploring urban functional zones based on spatio-temporal OD data and Points of Interest data. We first adaptively partition the territory into region units based on adaptive blue noise sampling method, extract POI feature matrix (location information) as a prior knowledge, model multidimensional spatio-temporal OD data as a tensor for addressing multifaceted features and propose an augment tensor-based algorithm that enables users to simultaneously combine mobility semantics and inherent location information for identifying functional zones. In addition, we design a set of visual encodings to better understand and interpret the results in a visual and intuitive manner. This system has been demonstrated using two case studies with a real-world dataset of HangZhou city and domain-expert interviews.

Senti-COVID19: An Interactive Visual Analytics System for Detecting Public Sentiment and Insights Regarding COVID-19 From Social Media

As governments take measures against COVID-19, the epidemic situation is expected to improve, but public sentiment is likely to fluctuate during this process, potentially influencing the best course of action. Social media has become a prevalent way for the public to express emotions and opinions in recent times. So that, the sentiment analysis on top of it may detect and provide valuable evidence of public attitude and help governments subsequent formulation of measures and policies. We present Senti-COVID19, an interactive visual analytic system for reflecting and analyzing public sentiment and detecting sentiment fluctuation triggers on social media. Senti-COVID19 adopts lexicon-based sentiment analysis to divulge the public opinion to COVID-19 events, employing libraries to extract keywords and statistics for providing detailed information. In addition, it offers visualizations for presenting the analysis, allowing users to quickly discover relevant information. Our results show that Senti-COVID19 can be used effectively to analyze sentiment from social media text, allowing users to explore derived data and identify insights from the collected tweets.

TaxThemis: Interactive Mining and Exploration of Suspicious Tax Evasion Groups.

Tax evasion is a serious economic problem for many countries, as it can undermine the government's tax system and lead to an unfair business competition environment. Recent research has applied data analytics techniques to analyze and detect tax evasion behaviors of individual taxpayers. However, they have failed to support the analysis and exploration of the related party transaction tax evasion (RPTTE) behaviors (e.g., transfer pricing), where a group of taxpayers is involved. In this paper, we present TaxThemis, an interactive visual analytics system to help tax officers mine and explore suspicious tax evasion groups through analyzing heterogeneous tax-related data. A taxpayer network is constructed and fused with the respective trade network to detect suspicious RPTTE groups. Rich visualizations are designed to facilitate the exploration and investigation of suspicious transactions between related taxpayers with profit and topological data analysis. Specifically, we propose a calendar heatmap with a carefully-designed encoding scheme to intuitively show the evidence of transferring revenue through related party transactions. We demonstrate the usefulness and effectiveness of TaxThemis through two case studies on real-world tax-related data and interviews with domain experts.

Visual analysis of retailing store location selection

An appropriate location of the retailing store is vital for achieving business success. However, a massive amount of complex information needs to be considered in location selection, such as customer flow, the business environment, and current business performance. Unlike the traditional location recommendation method of statistical sampling, we establish a model of business-district attractiveness based on customer flow. Besides, we build an interactive visual analysis system with a user-friendly interface for an interactive visual query about complex business and environment information. Our system can help users select retailing store locations, support interactive visual queries and display rich information to facilitate managers in decision-making.

TargetingVis: visual exploration and analysis of targeted advertising data

Targeted advertising is a dominant form of online advertising. It considers advertisers’ major concern of their customers, including the consumers’ certain traits, interests and individual preferences. To promote the effectiveness of advertisement delivery, advertising analysts need to understand advertiser delivery behavior and problems in targeting structure. However, statistical methods cannot meet analytical requirements completely, and analysts have to spend a lot of time reading countless data reports. Concretely, there is no efficient tool accomplishing analysis tasks such as exploring targeting usage at different levels, discovering useful or abnormal targeting combination patterns, finding competition from user behavior. In this paper, we design and implement an interactive visual analytics system named TargetingVis to visualize targeted advertising delivery data to face the challenges. After conducting a detailed requirements analysis with the domain experts from Tencent Inc., we design TargetingVis with four linked views: a novel chord diagram for cross-level exploration of targeting relations, a view for delving into the analysis of targeting combination patterns, an auxiliary view for displaying data indicators and a view to help gain insights into the behavior of advertisers. Finally, we evaluate the usability and efficiency through experiments based on real-world datasets.

SuPoolVisor: a visual analytics system for mining pool surveillance

Cryptocurrencies represented by Bitcoin have fully demonstrated their advantages and great potential in payment and monetary systems during the last decade. The mining pool, which is considered the source of Bitcoin, is the cornerstone of market stability. The surveillance of the mining pool can help regulators effectively assess the overall health of Bitcoin and issues. However, the anonymity of mining-pool miners and the difficulty of analyzing large numbers of transactions limit in-depth analysis. It is also a challenge to achieve intuitive and comprehensive monitoring of multi-source heterogeneous data. In this study, we present SuPoolVisor, an interactive visual analytics system that supports surveillance of the mining pool and de-anonymization by visual reasoning. SuPoolVisor is divided into pool level and address level. At the pool level, we use a sorted stream graph to illustrate the evolution of computing power of pools over time, and glyphs are designed in two other views to demonstrate the influence scope of the mining pool and the migration of pool members. At the address level, we use a force-directed graph and a massive sequence view to present the dynamic address network in the mining pool. Particularly, these two views, together with the Radviz view, support an iterative visual reasoning process for de-anonymization of pool members and provide interactions for cross-view analysis and identity marking. Effectiveness and usability of SuPoolVisor are demonstrated using three cases, in which we cooperate closely with experts in this field.

Multi-level and multi-perspective visual correlation analysis between general courses and program courses

Exploring the potential impact of important general courses on program-specific courses in universities can help to improve the entire teaching and learning process for an academic major. However, the large number of courses and multiple factors affecting students’ course grades makes it difficult to reveal and analyze the complicated relationship between the two types of courses only from a single perspective or at a single level. Thence, this paper starts with analysis of historical course grades data within an undergraduate program and then presents an interactive visual analytic system, MVCAS, which is designed to demonstrate and explore the various correlations between these two types of courses at different levels and from different perspectives. The major contributions of this work include: (1) a multi-angle preprocessing of course grades data, including decomposition, extraction and conversion; (2) multiple coordinated analysis views which make it possible to effectively explore the overall, categorical and pairwise course correlations and further link courses with students, instructors and semesters together; and (3) a top-down correlation analysis process for general courses and program ones. The effectiveness and usefulness of MVCAS have been preliminarily demonstrated through a case study, in which the field experts use this tool to investigate different levels of correlations between the focused mathematics and program-specific courses in a computer science major comprehensively.

PeckVis: A Visual Analytics Tool to Analyze Dominance Hierarchies in Small Groups.

The formation of social groups is defined by the interactions among the group members. Studying this group formation process can be useful in understanding the status of members, decision-making behaviors, spread of knowledge and diseases, and much more. A defining characteristic of these groups is the pecking order or hierarchy the members form which help groups work towards their goals. One area of social science deals with understanding the formation and maintenance of these hierarchies, and in our work we provide social scientists with a visual analytics tool - PeckVis - to aid this process. While online social groups or social networks have been studied deeply and lead to a variety of analyses and visualization tools, the study of smaller groups in the field of social science lacks the support of suitable tools. Domain experts believe that visualizing their data can save them time as well as reveal findings they may have failed to observe. We worked alongside domain experts to build an interactive visual analytics system to investigate social hierarchies. Our system can discover patterns and relationships between the members of a group as well as compare different groups. The results are presented to the user in the form of an interactive visual analytics dashboard. We demonstrate that domain experts were able to effectively use our tool to analyze animal behavior data.

A Visualization System for Performance Analysis of Image Classification Models

Developing machine learning models for image classification problems involves various tasks such as model selection, layer design, and hyperparameter tuning for improving the model performance. However, regarding deep learning models, insufficient model interpretability renders it infeasible to understand how they make predictions. To facilitate model interpretation, performance analysis at the class and instance levels with model visualization is essential. We herein present an interactive visual analytics system to provide a wide range of performance evaluations of different machine learning models for image classification. The proposed system aims to overcome challenges by providing visual performance analysis at different levels and visualizing misclassification instances. The system which comprises five views - ranking, projection, matrix, and instance list views, enables the comparison and analysis different models through user interaction. Several use cases of the proposed system are described and the application of the system based on MNIST data is explained. Our demo app is available at https://chanhee13p.github.io/VisMlic/.

EgoVis: A Visual Analysis System for Social Networks Based on Egocentric Research

The development of crowd intelligence makes the structure of social network more complex and changeable. Research on social network should be more in-depth and focus on the changes of structure. Ego-network, which represents the relationship between specific individual and the related people, is a hot issue among the research of dynamic social network. The evolution of ego-network is highly dynamic and pluralistic, it is hard to capture its evolutionary pattern over time. To help users analyze the individual characteristics and hidden patterns in multivariate ego-network, we present EgoVis, an interactive visual analysis system for exploring and analyzing complex structural relationships in dynamic network. Based on the task requirements of network evolution analysis, we propose a task taxonomy which is suitable for ego-network research and analysis, design novel visual fonts, and analyze the evolution of dynamic ego-network relations from the three dimensions: overview, subgroup, and detail-ego. Finally, the validity and practicability of EgoVis are verified on DBLP citation network dataset.

Motion Browser: Visualizing and Understanding Complex Upper Limb Movement Under Obstetrical Brachial Plexus Injuries.

The brachial plexus is a complex network of peripheral nerves that enables sensing from and control of the movements of the arms and hand. Nowadays, the coordination between the muscles to generate simple movements is still not well understood, hindering the knowledge of how to best treat patients with this type of peripheral nerve injury. To acquire enough information for medical data analysis, physicians conduct motion analysis assessments with patients to produce a rich dataset of electromyographic signals from multiple muscles recorded with joint movements during real-world tasks. However, tools for the analysis and visualization of the data in a succinct and interpretable manner are currently not available. Without the ability to integrate, compare, and compute multiple data sources in one platform, physicians can only compute simple statistical values to describe patient's behavior vaguely, which limits the possibility to answer clinical questions and generate hypotheses for research. To address this challenge, we have developed MOTION BROWSER, an interactive visual analytics system which provides an efficient framework to extract and compare muscle activity patterns from the patient's limbs and coordinated views to help users analyze muscle signals, motion data, and video information to address different tasks. The system was developed as a result of a collaborative endeavor between computer scientists and orthopedic surgery and rehabilitation physicians. We present case studies showing physicians can utilize the information displayed to understand how individuals coordinate their muscles to initiate appropriate treatment and generate new hypotheses for future research.

Finding communities in bicycle sharing system

The bicycle sharing system (BSS) provides a more sustainable transport paradigm in big cities. The recorded cycling trajectories can be used to detect human movement patterns. Community detection methods have been used to study BSS from a complex network perspective. However, the previous used modularity-based methods not only ignored the interdependencies of bicycle flows in the system, but also suffered from the problem of resolution limit. The in-depth analysis of detection results is also lacked. In this paper, we propose an interactive visual analytics system to detect the cycling communities of bicycle sharing system. Different kinds of community detection algorithms are adopted for finding station clusters; multiple inter-linked views are designed to visualize properties of the detected substructures from different perspectives. The real bicycle sharing dataset in Hangzhou is used for analysis, which demonstrates the effectiveness of our method. By using the system, analyzers can compare the cluster results generated by different algorithms, investigate the reason of the partition results based on different metrics, and find the relationship among human activity communities and the city subregional structures. This study provides insights into using bicycle sharing data to reveal human travel pattern and BSS usage pattern, which potentially aids in developing urban planning policies.

West Lake Tourist: A Visual Analysis System Based on Taxi Data

Mining the mobile pattern of the urban population plays an important role in city construction, and visual analysis is a powerful technique in studying mobile patterns. In this paper, based on the taxi trajectory data in Hangzhou, we share our design for an interactive visual analytic system, which helps analyzers leverage their domain knowledge to gain insight into travel patterns, including travel time rules of tourists and the distribution rules of pick-up and drop-off locations. Besides, our system can present the dynamic travel process and the Point of Interest (POIs) information of the origin and the destination. A case study has been conducted, which verifies that our system can provide tools for urban managers or urban experts on the design of scenic spot open entrances and exits and travel route planning.

A Visual Analytics System for Exploring, Monitoring, and Forecasting Road Traffic Congestion.

We present an interactive visual analytics system that enables traffic congestion exploration, surveillance, and forecasting based on vehicle detector data. Through domain expert collaboration, we have extracted task requirements, incorporated the Long Short-Term Memory (LSTM) model for congestion forecasting, and designed a weighting method for detecting the causes of congestion and congestion propagation directions. Our visual analytics system is designed to enable users to explore congestion causes, directions, and severity. Congestion conditions of a city are visualized using a Volume-Speed Rivers (VSRivers) visualization that simultaneously presents traffic volumes and speeds. To evaluate our system, we report performance comparison results, wherein our model is more accurate than other forecasting algorithms. We demonstrate the usefulness of our system in the traffic management and congestion broadcasting domains through three case studies and domain expert feedback.

AirInsight: Visual Exploration and Interpretation of Latent Patterns and Anomalies in Air Quality Data

Nowadays, huge volume of air quality data provides unprecedented opportunities for analyzing pollution. However, due to the high complexity, most traditional analytical methods focus on abstracting data, so these techniques discard the original structure and limit the understanding of the results. Visual analysis is a powerful technique for exploring unknown patterns since it retains the details of the original data and gives visual feedback to users. In this paper, we focus on air quality data and propose the AirInsight design, an interactive visual analytic system for recognizing, exploring, and summarizing regular patterns, as well as detecting, classifying, and interpreting abnormal cases. Based on the time-varying and multivariate features of air quality data, a dimension reduction method Composite Least Square Projection (CLSP) is proposed, which allows appreciating and interpreting the data patterns in the context of attributes. On the basis of the observed regular patterns, multiple abnormal cases are further detected, including the multivariate anomalies by the proposed Noise Hierarchical Clustering (NHC) method, abruptly changing timestamps by Time diversity (TD) indicator, and cities with unique patterns by the Geographical Surprise (GS) measure. Moreover, we combine TD and GS to group anomalies based on their underlying spatiotemporal correlations. AirInsight includes multiple coordinated views and rich interactive functions to provide contextual information from different aspects and facilitate a comprehensive understanding. In particular, a pair of glyphs are designed that provide a visual representation of the temporal variation in air quality conditions for a user-selected city. Experiments show that CLSP improves the accuracy of Least Square Projection (LSP) and that NHC has the ability to separate noises. Meanwhile, several case studies and task-based user evaluation demonstrate that our system is effective and practical for exploring and interpreting multivariate spatiotemporal patterns and anomalies in air quality data.