A system for visualizing sound source using augmented reality

Abstract

PurposeThe purpose of this paper is to design and propose a new interface for hearing‐impaired for the users who can hardly realize the environmental sound.Design/methodology/approachThe authors propose the use of an augmented reality (AR) system with sound source recognition to augment human vision. In this system, sound source and position is detected by using acoustic processing.FindingsThe authors confirmed that the source and direction of sound could be effectively recognized, and that AR was implemented, and thus that the user could use this system to recognize and visualize environmental sounds. When there was only a single sound source in the surrounding environment such as at home or when doing some simple work, and especially when a source was near a user, this system provided information on the sound source and visualized the sound source to satisfy the user's need.Originality/valueThe system can recognize the environmental sound in realtime and inform the user of the type of sound by showing a virtual object in the user's sight. Furthermore, the user can find the direction of the sound source by using a microphone array and locate the sound source through the AR marker attached to the object.