Touching Light: A Framework for the Facilitation of Music-Making in Mixed Reality

Abstract

Drawing upon the historical development of analog and digital technologies alongside the proliferation of computer-assisted performance practices, this research seeks to develop a framework for integrating Mixed Reality applications to live musical performance, specifically through the creation of a Microsoft HoloLens 2 Mixed Reality application in order to facilitate a live performance of an original musical composition for percussion and real-time Mixed Reality environment. Mixed Reality enables a performer to interact with virtual (holograms, VSTs, etc.) and physical (vibraphone, tuned drums, microphones, etc.) objects simultaneously. Tandem to the development of the conceptual framework was the composition of an original score for solo percussionist and Mixed Reality environment. The score was composed to uniquely serve this interaction between virtual and physical spaces by employing a custom HoloLens 2 application. A proposal of idiomatic performance practices as a ‘framework’ for music-making in Mixed Reality functions as the focus of this project. This framework was derived from iterative experimentation with the Microsoft HoloLens 2 and facilitated by the author’s practical music performance experience.