Interactive Visual Data Exploration Research Articles

Visual Analytics: Transferring, Translating and Transforming Knowledge from Analytics Experts to Non-technical Domain Experts in Multidisciplinary Teams

Today’s complex problems call for multidisciplinary analytics teams comprising of both analytics and non-technical domain (i.e. subject matter) experts. Recognizing the difference between data visualisaion (DV) (i.e. static visual outputs) and visual analytics (VA) (i.e. a process of interactive visual data exploration, guided by user’s domain and contextual knowledge), this paper focuses on VA for non-technical domain experts. By seeking to understand knowledge sharing from VA experts to non-technical users of VA in a multidisciplinary team, we aim to explore how these domain experts learn to use VA as a thinking tool, guided by their knowing-in-practice. The research described in this paper was conducted in the context of a long-term industry-wide research project called the ‘Visual Historical Atlas of the Australian Co-operatives’, led by a multidisciplinary VA team who faced the challenge tackled by this research. Using Action Design Research (ADR) and the combined theoretical lens of boundary objects and secondary design, the paper theorises a three-phase method for knowledge transfer, translation and transformation from VA experts to domain experts using different types of VA-related boundary objects. Together with the proposed set of design principles, the three-phase model advances the well-established stream of research on organizational use of analytics, extending it to the emerging area of visual analytics for non-technical decision makers.

Collaborative filtering over evolution provenance data for interactive visual data exploration

In interactive visual data exploration, users rely on recommendations on what data to explore next. EVLIN is a system that recommends queries to retrieve these data for the next exploration step, paired with suited visualizations. This paper extends EVLIN by combining its content-based recommendations with recommendations leveraging collaborative filtering to improve the effectiveness of recommendation-based visual data exploration. The recommendations rely on evolution provenance, which tracks users’ interactions during interactive visual data exploration. As more users explore a dataset, the evolution provenance of individual user explorations is incrementally integrated into a multi-user graph, for which we present match and merge algorithms. To compute collaborative-filtering recommendations, we present a search algorithm and optimizations to efficiently search queries similar to a current user’s query in the multi-user graph and give preference to queries that have been previously explored in an exploration step succeeding those similar queries. Our experimental evaluation studies the efficiency and effectiveness of the solutions proposed in this paper and demonstrates that using the full system with both content-based and collaborative-filtering recommendations enabled allows for effective interactive visual data exploration.

Interactive visual data exploration with subjective feedback: an information-theoretic approach

Visual exploration of high-dimensional real-valued datasets is a fundamental task in exploratory data analysis (EDA). Existing projection methods for data visualization use predefined criteria to choose the representation of data. There is a lack of methods that (i) use information on what the user has learned from the data and (ii) show patterns that she does not know yet. We construct a theoretical model where identified patterns can be input as knowledge to the system. The knowledge syntax here is intuitive, such as “this set of points forms a cluster”, and requires no knowledge of maths. This background knowledge is used to find a maximum entropy distribution of the data, after which the user is provided with data projections for which the data and the maximum entropy distribution differ the most, hence showing the user aspects of data that are maximally informative given the background knowledge. We study the computational performance of our model and present use cases on synthetic and real data. We find that the model allows the user to learn information efficiently from various data sources and works sufficiently fast in practice. In addition, we provide an open source EDA demonstrator system implementing our model with tailored interactive visualizations. We conclude that the information theoretic approach to EDA where patterns observed by a user are formalized as constraints provides a principled, intuitive, and efficient basis for constructing an EDA system.

Analysis and Visualization of Coastal Ocean Model Data in the Cloud

The traditional flow of coastal ocean model data is from High-Performance Computing (HPC) centers to the local desktop, or to a file server where just the needed data can be extracted via services such as OPeNDAP. Analysis and visualization are then conducted using local hardware and software. This requires moving large amounts of data across the internet as well as acquiring and maintaining local hardware, software, and support personnel. Further, as data sets increase in size, the traditional workflow may not be scalable. Alternatively, recent advances make it possible to move data from HPC to the Cloud and perform interactive, scalable, data-proximate analysis and visualization, with simply a web browser user interface. We use the framework advanced by the NSF-funded Pangeo project, a free, open-source Python system which provides multi-user login via JupyterHub and parallel analysis via Dask, both running in Docker containers orchestrated by Kubernetes. Data are stored in the Zarr format, a Cloud-friendly n-dimensional array format that allows performant extraction of data by anyone without relying on data services like OPeNDAP. Interactive visual exploration of data on complex, large model grids is made possible by new tools in the Python PyViz ecosystem, which can render maps at screen resolution, dynamically updating on pan and zoom operations. Two examples are given: (1) Calculating the maximum water level at each grid cell from a 53-GB, 720-time-step, 9-million-node triangular mesh ADCIRC simulation of Hurricane Ike; (2) Creating a dashboard for visualizing data from a curvilinear orthogonal COAWST/ROMS forecast model.

A Constrained Randomization Approach to Interactive Visual Data Exploration with Subjective Feedback

Data visualization and iterative/interactive data mining are growing rapidly in attention, both in research as well as in industry. However, while there are a plethora of advanced data mining methods and lots of works in the field of visualization, integrated methods that combine advanced visualization and/or interaction with data mining techniques in a principled way are rare. We present a framework based on constrained randomization which lets users explore high-dimensional data via ‘subjectively informative’ two-dimensional data visualizations. The user is presented with ‘interesting’ projections, allowing users to express their observations using visual interactions that update a background model representing the user's belief state. This background model is then considered by a projection-finding algorithm employing data randomization to compute a new ‘interesting’ projection. By providing users with information that contrasts with the background model, we maximize the chance that the user encounters striking new information present in the data. This process can be iterated until the user runs out of time or until the difference between the randomized and the real data is insignificant. We present two case studies, one controlled study on synthetic data and another on census data, using the proof-of-concept tool SIDE that demonstrates the presented framework.

Visual analytics in cheminformatics: user-supervised descriptor selection for QSAR methods.

BackgroundThe design of QSAR/QSPR models is a challenging problem, where the selection of the most relevant descriptors constitutes a key step of the process. Several feature selection methods that address this step are concentrated on statistical associations among descriptors and target properties, whereas the chemical knowledge is left out of the analysis. For this reason, the interpretability and generality of the QSAR/QSPR models obtained by these feature selection methods are drastically affected. Therefore, an approach for integrating domain expert’s knowledge in the selection process is needed for increase the confidence in the final set of descriptors.ResultsIn this paper a software tool, which we named Visual and Interactive DEscriptor ANalysis (VIDEAN), that combines statistical methods with interactive visualizations for choosing a set of descriptors for predicting a target property is proposed. Domain expertise can be added to the feature selection process by means of an interactive visual exploration of data, and aided by statistical tools and metrics based on information theory. Coordinated visual representations are presented for capturing different relationships and interactions among descriptors, target properties and candidate subsets of descriptors. The competencies of the proposed software were assessed through different scenarios. These scenarios reveal how an expert can use this tool to choose one subset of descriptors from a group of candidate subsets or how to modify existing descriptor subsets and even incorporate new descriptors according to his or her own knowledge of the target property.ConclusionsThe reported experiences showed the suitability of our software for selecting sets of descriptors with low cardinality, high interpretability, low redundancy and high statistical performance in a visual exploratory way. Therefore, it is possible to conclude that the resulting tool allows the integration of a chemist’s expertise in the descriptor selection process with a low cognitive effort in contrast with the alternative of using an ad-hoc manual analysis of the selected descriptors.Graphical abstractVIDEAN allows the visual analysis of candidate subsets of descriptors for QSAR/QSPR. In the two panels on the top, users can interactively explore numerical correlations as well as co-occurrences in the candidate subsets through two interactive graphs.Electronic supplementary materialThe online version of this article (doi:10.1186/s13321-015-0092-4) contains supplementary material, which is available to authorized users.

Hypothesis Generation in Climate Research with Interactive Visual Data Exploration

One of the most prominent topics in climate research is the investigation, detection, and allocation of climate change. In this paper, we aim at identifying regions in the atmosphere (e.g., certain height layers) which can act as sensitive and robust indicators for climate change. We demonstrate how interactive visual data exploration of large amounts of multi-variate and time-dependent climate data enables the steered generation of promising hypotheses for subsequent statistical evaluation. The use of new visualization and interaction technology--in the context of a coordinated multiple views framework--allows not only to identify these promising hypotheses, but also to efficiently narrow down parameters that are required in the process of computational data analysis. Two datasets, namely an ECHAM5 climate model run and the ERA-40 reanalysis incorporating observational data, are investigated. Higher-order information such as linear trends or signal-to-noise ratio is derived and interactively explored in order to detect and explore those regions which react most sensitively to climate change. As one conclusion from this study, we identify an excellent potential for usefully generalizing our approach to other, similar application cases, as well.