Applying machine learning to automated information graphics generation

Abstract

Information graphics, which include graphs, charts, and diagrams, are visual illustrations that facilitate human comprehension of information. In this paper, we present our work on applying machine learning to the automated generation of information graphics. Our approach is embodied in a hybrid graphics generation system, IMPROVISE*, which uses both rule-based and example-based generation engines. We discuss the use of machine learning to support such systems from three aspects. First, we introduce an object-oriented, integrated hierarchical feature representation for annotating information graphics. Second, we describe how to use decision-tree learning to automatically extract design rules from a set of annotated graphic examples. Our results demonstrate that we can acquire, with quantitative confidence, concise rules that are difficult to obtain in handcrafted rules. Third, we present a casebased learning method to retrieve matched graphic examples based on user requests. Specifically, we use a semantics-based, quantitative visual similarity measuring algorithm to retrieve the top-k matched examples from a visual database. To help users browse a graphics database and understand the inherent relations among different examples, we combine our similarity measuring model with a hierarchical clustering algorithm to dynamically organize our graphic examples based on user interests.