Review: Ludified, edited by Marko Ciciliani, Barbara Lüneburg, and Andreas Pirchner

Abstract

Ludified is an important new text edited by Marko Ciciliani, Barbara Lüneburg, and Andreas Pirchner that provides a comprehensive documentation of artistic research carried out as part of the GAPPP—Gamified Audiovisual Performance & Performance Practice project. GAPPP was funded by the Austrian Science Fund and carried out from 2016 to 2020 at the Institute of Electronic Music and Acoustics of the University of Music and Performing Arts Graz. In recent years the use of video games technologies in contemporary music has seen significant growth, as composers have explored the possibilities of video game engines, virtual reality (VR), and other associated interactive technologies in their work. This has arisen because modern game engines provide powerful and readily accessible tools for generating interactive audiovisual experiences, but also because video game culture has become so firmly entrenched in society that its territories and modalities are ripe for artistic investigation. As a book that provides a detailed exploration into this area, Ludified is both welcome and timely, and it will interest audiences and practitioners working in this area.Ludified actually comprises two books in one, which are alternated by flipping over the weighty tome and reading from the opposite side. The “blue book” explores a spectrum of work from the GAPPP project, while the “green book” focuses on compositions and discourse by project leader Ciciliani. The production quality of Ludified is excellent, including color printing and a USB drive/bookmark with 32 GB of high-definition video files documenting the performances. The latter is a valuable addition, allowing the reader to watch and listen in parallel with reading. Short of attending the concerts in person, the combination of text and video gives a good impression of the works, and as a way to successfully publish the results of practice-led research, Ludified is exemplary. There is also an accompanying augmented reality app that can be downloaded; however, a minor limitation is that no actual software is included on the USB drive—I found myself wanting to run some of the applications myself to investigate further, but including these would inevitably have presented significant challenges in terms of compatibility, and in many cases specialist hardware would be needed to explore the performance system being presented, so the decision to focus on video documentation makes sense.With the exception of some works that have alternate modes of presentation as installations, the artistic works documented in Ludified are mostly located as audiovisual compositions conceived for performance onstage to an audience. They investigate a range of compositional strategies that implement gamified approaches. In the blue book, Christof Ressi’s game over provides a 2D open world with a performance on clarinet, where motion sensors allow the performer to control the game whilst also carrying out free improvisation. Offering a different approach, Ressi’s piece Terrain Study is a composition for a solo clarinet performance situated in a 3D landscape in VR, where movement over the terrain affects the sound of the instrument, and collisions with spherical objects trigger granular synthesis. Martina Menegon and Stefano D’Alessio’s To Kill Two Birds with One Stone combines Pong and Rock, Paper, Scissors into a single game that is controlled via hand gestures captured with Leap Motion controllers. Attractive Correlations by Kosmas Giannoutakis creates an interactive system for agents using multiple loudspeakers and microphones. Rob Hamilton’s Trois Machins de la Grâce Aimante is an elaborate work exploring the idea of a string quartet in VR via his Coretet virtual instrument, built using Unreal and Pure Data. String Mask Overflow by Pedro González Fernández responds to Hamilton’s piece via a multimedia work for violin, movement, shadow, video, and electronics. This book also features a lecture by Olli Tapio Leino that reflects on responses to the “boring” aspects of games such as Euro Truck Simulator 2, which Alyssa Aska responds to with The Missing Piece, a composition in which clarinet controls a game character wandering across a desolate plane.Focusing on Ciciliani’s output, the green book discusses Tympanic Touch, a multisensory performance in a game system, exploring the sounds and textures of various materials. By Ciciliani’s own admission, Tympanic Touch is the least overtly gamified composition on offer yet embeds competitive play through the actions of the performers. Atomic Etudes and Chemical Etudes are two works for Monome that evoke retro game aesthetics through the luminous grid interface of the interactive controller and generated sounds. Engaging with the visual languages of more recent open-world games, Kilgore and the installation version Kilgore’s Resort allow characters to navigate across a 3D landscape in which their interactions elicit an audiovisual performance. Lastly, Anna & Marie is an audiovisual work based on the lives of two eighteenth-century anatomists who were pioneers in the field of ceroplastics (wax sculpting). This composition integrates nonlinear storytelling, 3D virtual worlds, violins, and electronics and has an augmented reality version, which can be experienced by downloading and viewing parts of the book with a special app.As a whole, the works from both the green and blue books invoke our imagination of new possibilities for audiovisual performance, which emerge through the use and repurposing of video game technologies. The associated chapters give important insights into the compositional methodologies used to realize the works. Detailed discussions of the audiovisual relationships and interactive designs employed are provided, which give the reader a clearer understanding of the thinking behind them. These discussions have also been effectively balanced in Ludified with a good amount of technical detail, which other practitioners will find informative. For example, in the discussion of Hamilton’s Trois Machins de la Grâce Aimante, he gives details of production setup aspects such as the networking of several computers, providing useful insights into the practical realization of the project at concerts.Besides exploring compositional strategies, Ludified directs significant attention to the audience experience and perception of these works. In particular, Ciciliani’s polar diagram allows us to consider dimensions of gamified composition and performance, related to various aspects such as the amount of agency the system provides, the required level of expertise, and others. As evidenced through the analysis of works produced as part of the GAPPP project, this model has a clear utility for composing and analyzing gamified works. Interesting points about audience experience also emerge from the discussion of the works and audience research in the respective chapters. For instance, Ressi provides some insightful commentary on deliberately keeping bugs in the game code to promote the possibilities of unexpected and chaotic events, which I found to be very revealing.In general, the audiovisual works in GAPPP that utilize 3D game engines are striking in their treatment of virtual landscapes as playgrounds for sonic and visual experimentation. On the one hand, the composers are working with the visual languages and mechanisms that provide the substance of video games, but they seem to do so in a way that avoids the polished veneer that we are so used to seeing in commercial video games. Instead, the game engine is repurposed; fragments of 3D models, symbols, and interactive mechanisms are dissolved, reconfigured, and reimagined. This is refreshing—in a world where we increasingly inhabit highly commodified, restrictive digital spaces, these audiovisual performances help us reimagine virtual worlds as places of playful improvisation, freeness, and spontaneity.While Ludified presents a very successful body of work, the authors do not shy away from casting a critical eye over some of the mechanisms that are explored. For example, Lüneburg observes a disjunct between the audience and performer experience in Hamilton’s Trois Machins de la Grâce Aimante, where the audience only sees a projected representation of what the performer sees in the headset. There seems to be some fascination for audiences in the theatricality of watching others perform in VR, observing them wearing these strange masks that engulf their senses and capture them in a digital trance. Ludified also draws attention to the prevalence of e-sports as an indication that performances by others within game engines find a significant audience. Yet for interactive game technologies, there remains a feeling that audiences could be missing out on something when they are situated as passive observers rather than participants in the “game” that is being played. Installations can overcome this, but this may come at the expense of seeing/hearing the game being played as an instrument by a virtuoso, in the way the composer intends. It is to the credit of Ludified and the GAPPP project that it stimulates thought on important issues such as these.Overall, Ludified is a rich and varied text, with many insights to offer on the composition, performance, and perception of gamified audiovisual works in contemporary music. Just as the GAPPP project symposium employed a proposal/response format, where lectures provided a “proposal,” to which artistic performances then provided a “response” (and vice versa), Ludified can be read as a set of proposals that invite further responses from artists and researchers working in this exciting field. I look forward to seeing what comes next.