Gesture and Voice Recognition in Story Telling Application

Abstract

Stories are one of the significant things that everyone listens to during childhood. Not only do they give lessons about ethics and morals but also help improving comprehension, enunciation and learning amongst the children. Thus stories play an important role in strengthening the learning of a child at a very early age. The Story Telling application developed is one such technique that allows to create and narrate stories with the help of gesture and voice recognition techniques. This can serve as a tool for the teachers in order to make the learning experience more enjoyable and also the children can create their own stories which in turn will help them widen their imagination. This paper discusses how the voice and gesture modules have been used to aid the process of storytelling. Gesture recognition is used to detect the hand gestures which are used to move the characters while the story is being narrated using the blob detection. While voice recognition is used for changing the background dynamically during story's narration according to the words used in the story by matching with the database which stores a mapping of the words and corresponding images. Thus this paper gives an overview of the integration of gesture and voice recognition used to implement the storytelling application.