A New 3D Augmented Reality Application for Educational Games to Help Children in Communication Interactively

Abstract

In recent years, the use of technology to help children for augmented and alternative communication (AAC) is extremely a vital task. In this paper, a novel three-dimensional human-computer interaction application is presented based on augmented reality (AR) technology for assisting children with special problems in communication for social innovation. To begin with, three-dimensional human hand model is constructed to estimate and track the hand’s position of users. An extended particle filter is applied for calculating the pose of background and the positions of children. The likelihoods based on the edge map of the image and pixel color values are utilized to estimate the joint likelihood in three-dimensional model. A flexible real-time hand tracking framework using the ‘golden energy’ scoring function is integrated for capturing region of interests. An inertial tracking technique is used for calculating the quaternion. Three-dimensional models from Google SketchUp are employed. We then use a built QR-code for scanning to access the system, and then utilize for selecting a character three-dimensional designed cartoon by applying the Vidinoti image application. After that, representative three-dimensional cartoons and augmented environments are overlaid, so that it is able to entertain children. A printed coloring photo, called Augmented Flexible Tracking is designed and provided in the system for visualization. The process of the system is done in real-time. Our experiments have revealed that the system is beneficial both quantitatively and qualitatively for assisting children with special needs in communication interactively.