CLE-UMA – A Creative Learning Environment Using Multimodal Analogies

Abstract

In this paper, we present a novel approach for applying multimodal analogies to creative learning scenarios. We have chosen Japanese language learning, in particular kanji acquisition, as the first application domain for our research. Our solution is based on a kanji dictionary which we enriched with visual, compositional, and semantic information. We have manually assigned images to 3,500 kanji and elaborately annotated the mappings from the lexical data to its visual representation. This knowledge enables us to find high-quality analogies for associations between the visual and textual dimensions. The analogies are integrated into our Web-based contextual language learning environment to empower several creative learning applications through augmented browsing technology including exercises, quizzes, and educational games.