From Sketching to Natural Language

Abstract

Data visualization is the primary means by which data analysts explore patterns, trends, and insights in their data. Unfortunately, existing visual analytics tools offer limited expressiveness and scalability when it comes to searching for visualizations over large datasets, making visual data exploration labor-intensive and timeconsuming. We first discuss our prior work on Zenvisage that helps accelerate exploratory data analysis via an interactive interface and an expressive visualization query language, but offers limited flexibility when the pattern of interest is under-specified and approximate. Motivated from our findings from Zenvisage, we develop ShapeSearch, an efficient and flexible pattern-searching tool that enables the search for desired patterns via multiple mechanisms: sketch, natural-language, and visual regular expressions. ShapeSearch leverages a novel shape querying algebra that can express a large class of shape queries and supports query-aware and perceptually-aware optimizations to execute shape queries within interactive response times. To further improve the usability and performance of both Zenvisage and ShapeSearch, we discuss a number of open research problems.