Audio Augmented Reality in Smart Learning Environments

Abstract

Image processing-based augmented reality (AR) is widely used in several fields. However, audio content is also crucial in certain cases, for example, focusing on appreciating artwork in a museum, rather than a virtually synthesized image. In this study, an attempt was made to provide ubiquitous learning (u-learning) services by using audio AR for smart cities where people do not need smartphones, tablets, or wearable devices for learning. The concept of “unmanned stores” was introduced in the proposed learning environments. A 360° camera was mounted on the ceiling of an exhibition room. An algorithm was used to detect moving objects and locate a visitor, from whose position an information-processing platform searched the showpiece nearest to the visitor and selected the corresponding audio guide. For multiple visitors, infrared (IR) sensors were used to count the visitors inside the room. A loudspeaker array was built to emit directional sounds for eliminating interference between multiple sound sources. Along with the proposed algorithm for increasing the accuracy while sensing multiple movements, the conventional sound field synthesis method was modified to improve the listening experience and learning quality. Image-based sensor technology, IR detection, and an audio signal-processing platform were integrated into a smart environment in which people can conduct u-learning without personal devices. The method can provide immersive audio tours with virtual narrators and peer-helpers in schools, museums, and art galleries. Qualitative and quantitative analyses indicated that effective hands-free and eyes-free learning was achieved.