Sound of Vision – 3D Scene Reconstruction from Stereo Vision in an Electronic Travel Aid for the Visually Impaired

Abstract

The paper presents the preliminary results for the parametrization of 3D scene for sonification purposes in an electronic travel aid (ETA) system being built within the European Union’s H2020 Sound of Vision project. The ETA is based on the concept of sensory substitution, in which visual information is transformed into either acoustic or haptic stimuli. In this communication we concentrate on vision-to-audio conversion i.e. employing stereovision for reconstruction of 3D scenes and building a spatial model of the environment for sonification. Two prerequisite approaches for the sonification are proposed. One involves the direct sonification of the so-called “U-disparity” representation of the depth map of the environment, while the other relies on the processing of the depth map to extract obstacles present in the environment and presenting them to the user as auditory icons reflecting specific size and location of the sonified object.