Evolving the Ideal Visual-to-Auditory Sensory Substitution Device Using Interactive Genetic Algorithms

Abstract

Visual-to-auditory sensory substitution devices have various benefits over tactile counterparts (eg, less hardware limitations), but they also suffer from several drawbacks (eg, learning time, potentially unpleasant sounds). An ‘ideal’ device would be intuitive to learn, pleasant to listen to, and capture relevant visual information in sufficient detail. In this presentation, we outline the general problem of how to convert an image into sound, and we give an overview of some possible approaches to the problem. We then go on to describe our own recent explorations using Interactive Genetic Algorithms (IGAs). IGAs enable a highly dimensional problem space to be explored rapidly. Initially, a set of orthogonally varying settings need to be identified (eg, different levels of maximum and minimum pitch, different ways of mapping lightness-loudness, musical vs non-musical intervals), and a set of random permutations of these settings are chosen. Participants then evaluate the ‘fitness’ of these different algor...