Local context-based recognition of sketched diagrams

Abstract

We present a new methodology aimed at the design and implementation of a framework for sketch recognition enabling the recognition and interpretation of diagrams. The diagrams may contain different types of sketched graphic elements such as symbols, connectors, and text. Once symbols are distinguished from connectors and identified, the recognition proceeds by identifying the local context of each symbol. This is seen as the symbol interface exposed to the rest of the diagram and includes predefined attachment areas on each symbol. The definition of simple constraints on the local context of each symbol allows to greatly simplify the definition of the visual grammar, which is used only for further refinement and interpretation of the set of acceptable diagrams. We demonstrate the potential of the methodology using flowcharts and binary trees as examples.