Enabling instructors to develop sketch recognition applications for the classroom

Abstract

Instructors and students sketch graphical diagrams in a variety of classes from pre-K through higher education. Hand sketching the diagrams can engage students' creative processes as they watch the diagrams being created in real-time. Animations can help a students' functional understanding. However, hand-sketched diagrams currently remain static and uninterpreted, and animations currently have to be canned pre-made diagrams. Sketch recognition systems recognize hand drawn diagrams, but they take a lot of time and effort to build and require expertise in sketch recognition programming. To simplify the creation of sketch recognition system, we have built LADDER, a language to describe how shapes in a domain are drawn, displayed, and edited for use in sketch recognition, and GUILD, a system to automatically generate user interfaces from LADDER descriptions. The goal of this work is to facilitate the development of sketch recognition systems to allow non-experts in sketch recognition systems, such as teachers develop sketch systems for their classroom. The research is continuously being improved, but thus far, over twenty people have built sketch recognition systems using these technologies.