Motion Tracking of a Fish as a Novel Way to Control Electronic Music Performance

Abstract

The authors have mapped the three-dimensional motion of a fish onto various electronic music performance gestures, including loops, melodies, arpeggio and DJ-like interventions. They combine an element of visualization, using an LED screen installed on the back of an aquarium, to create a link between the fish’s motion and the sonified music. This visual addition provides extra information about the fish’s role in the music, enabling the perception of versatile and developing auditory structures during the performance that extend beyond the sonification of the momentary motion of objects.