Sound Field Projection System using Optical See-Through Head Mounted Display

Abstract

There are various ways to grasp the spatial and temporal structures of sound field. Sound field visualization is an effective technique to understand spatial sound information. For example, acoustical holography, optical methods, and beam-forming have been proposed and studied. In recent years, augmented reality (AR) technology has rapidly developed and is now more familiar. Many sensors, display devices, and ICT technologies have been implemented in AR equipment, which enable interaction between real and virtual worlds. In this paper, we propose an AR display system, which displays the results obtained by the beam-forming method. The system consists of 16ch microphone array, real-time sound field visualization system and optical see-through head mounted display (OST-HMD). Real-time sound field visualization system analyses sound signals recorded by 16ch microphone array by beam-forming method. Processed sound pressures data are sent to OST-HMD by using transmission control protocol (TCP), and colormap is projected on real world. Settings property of real-time sound field visualization system can be changed by using virtual user interface (UI) and TCP. In addition, multi-users can experience the system by sharing sound pressures and settings property data. Using this system, users wearing OST-HMD can observe sound field information intuitively.