Transmedia Choreography: Integrating Multimodal Video Annotation in the Creative Process of a Social Robotics Performance Piece

Stephan Jürgens

doi:10.16995/bst.44

Abstract

In this article I explore my current choreographic work with human and flying robotic performers. I adopt a number of existing conceptual lenses for the analysis of my choreographic approach, including ‘interdisciplinary choreography’, ‘post-choreography’, ‘processing choreography’, ‘generative techniques’ and ‘metacreation’. Following a brief discussion of choreographic practices in digital (live) performance environments, I propose two new terms: ‘transmedia choreography’ and ‘transmedia score’. Examples of multimodal video annotation in the creative process of our recent social robotics performance piece I-Care-Us will serve to discuss novel approaches in the production process of current transmedial performance works.

Highlights

In this article I explore my current choreographic work with human and flying robotic performers
Problems in choreographic analysis arise when interactive system design comes into play, posing questions such as: how do you choreograph an interaction between a human performer and a semi-autonomous agent, for instance, a flying robot programmed to make decisions, which may not be repeatable? Is the motion design of the robot’s behaviors choreography, or something different? What kind of notation or documentation could support such choreographic practices? Building on existing novel conceptual frameworks that re-evaluate the term choreography in the context of digital performance, I will propose two new terms and exemplify their utilization with reference to the work of several well-known practitioners in the field as well as illustrating my own choreographic approach
Paraphrasing the title of this important article, one could ask: what existed before postchoreography? According to Birringer, the ‘conventional organization and articulation of dance as choreographic practice relied on a coding that fixes the steps and sequences of movement and makes them repeatable’ (Birringer, 2008a: 118)

Summary

A Transmedia Knowledge Base for Contemporary Dance

The Transmedia Knowledge Base for Contemporary Dance Research Project (TKB, 2010-2013) has come to a close. Coordinated by linguist Carla Fernandes (Universidade Nova de Lisboa, FCSH, Portugal), the TKB-project aimed at the design and construction of an open-ended multimodal knowledge base to document, annotate and support the creation of contemporary dance pieces. A specialist-based approach can offer a number of advantages: individual working hours can be seamlessly integrated as each collaborator takes their own notes and publishes them to a shared platform; complex sections of the piece can be identified as the different specialists annotate their observations on different tracks connected to the same video footage; and the need for discussion between collaborators becomes more obvious as a consequence of the respective annotations Along this line of inquiry we are currently working on a case study investigating whether the Creation Tool (and the use we are able to make of it) can support efficient communication between the sound designer (Simão Costa) and the choreographer (myself). The development of tools such as the TKB Creation Tool prototype allow sophisticated multidimensional connections across often disparate information blocks of complex digital (live) performance work and transmedia choreography

Conclusion

At the international conference Multimodal Communication