Mocap Technology Research Articles

Accepting Human-like Avatars in Social and Professional Roles

Humans report perceptions of unease or eeriness as humanoid/android robots and digital avatars approach human-like physical resemblance, a phenomenon alluded by the Uncanny Valley theory. This study extends the discussions on interactions and acceptance of digital avatars with findings from three experiments. In the first, perceptive evaluation of actors in clips from computer-generated animation and a live-action version of the same movie was examined. In the second experiment, we considered short clips with highly realistic digital avatars to measure recognition ability, the extent of eeriness, and specific physical features identified as unreal. The fixation area and pupil size variation recorded using an eye tracker were analyzed to infer attention to the body, face, and emotional response, respectively. Building on these findings, the third experiment looked at acceptance in roles requiring human skill, empathy, and cognitive ability. The results show that based on perceptions from physical attributes, the eeriness scores diverge from the uncanny valley theory as human-likeness increases. The realistic CGI and mocap technology could have helped cross the valley. Visual attention inferred from gaze behavior was similar for live-action and CGI. At the same time, we observe pupil size changes reflecting emotions like eeriness when the avatars either talked or smiled. Proficiency and acceptance scores were lower for roles requiring complex social cognition processes, such as friends and judicial decision-making. Interestingly, real-life stereotypes of gender roles were transferred to digital avatars too. The findings suggest an ambiguity in accepting human-like avatars in social and professional interactions, emphasizing the need for a multi-dimensional approach when applying the uncanny valley theory. A detailed and contextual examination is imperative as technological advancements have placed humans closer to co-existing with digital or physical android/humanoid robots.

Local anatomically-constrained facial performance retargeting

Generating realistic facial animation for CG characters and digital doubles is one of the hardest tasks in animation. A typical production workflow involves capturing the performance of a real actor using mo-cap technology, and transferring the captured motion to the target digital character. This process, known as retargeting , has been used for over a decade, and typically relies on either large blendshape rigs that are expensive to create, or direct deformation transfer algorithms that operate on individual geometric elements and are prone to artifacts. We present a new method for high-fidelity offline facial performance retargeting that is neither expensive nor artifact-prone. Our two step method first transfers local expression details to the target, and is followed by a global face surface prediction that uses anatomical constraints in order to stay in the feasible shape space of the target character. Our method also offers artists with familiar blendshape controls to perform fine adjustments to the retargeted animation. As such, our method is ideally suited for the complex task of human-to-human 3D facial performance retargeting, where the quality bar is extremely high in order to avoid the uncanny valley, while also being applicable for more common human-to-creature settings. We demonstrate the superior performance of our method over traditional deformation transfer algorithms, while achieving a quality comparable to current blendshape-based techniques used in production while requiring significantly fewer input shapes at setup time. A detailed user study corroborates the realistic and artifact free animations generated by our method in comparison to existing techniques.

Performing without the use of a net: Making an animated feature without a storyboard

Abstract Generally we think that the storyboard is the holy grail when making animation movies. But that same storyboard can also be an obstacle to improvisation, intuition, free artistic expression and last-minute ideas. Reflecting on this dilemma, I devised an alternative way to make an animated feature based on mocap technology and the talent of a cast of professional actors, skipping the storyboard entirely and having the freedom for changes almost until rendering. Cafard (2015) is that experiment. It is on the crossroad between theatre, cinema and game technology, combining the best of each world.

Stillness, breath and the spine - Dance performance enhancement catalysed by the interplay between 3D motion capture technology in a collaborative improvisational choreographic process

We propose that Motion Capture (MoCap) in dance is part of, but distinct from, the traditional use of film or video as an archival and indexical instrument. Furthermore in the move from the recording of framed and temporally consistent slices of linear film or video ‘footage’ to the collection of profoundly mutable digital data utilising an Omniscient Frame, there is a fundamental conceptual shift in the creative re-shaping of the performance through a shared choreographic process. In the improvised choreographic and live performance process, qualitative differences in evaluating MoCap were identified. Study #1 (2015) is a collaborative screendance work by the authors, dancer (Author One), and digital artist (Author two). This piece utilises 3D Motion Capture technology and 3D digital animation software as part of a series of dance and moving image experiments. MoCap offers enhanced perspectives towards compositional awareness and evaluation between live and digital platforms. The human movement material produced in response to the MoCap technology optimises the potential of the technology and the human, moving body, with a catalysing force. We propose that what is transferred from live to digital via the omniscient motion capture camera informs what we see towards creative possibilities. We identify that the live performer as movement data does manifest as digital presence. We propose that we can view dance data and that this does aid performance – not towards quantitative evaluation but in capturing specific, human movement qualities towards qualitative artistic evaluation and critique.

Natural and hybrid bimanual interaction for virtual assembly tasks

This paper focuses on the simulation of bimanual assembly/disassembly operations for training or product design applications. Most assembly applications have been limited to simulate only unimanual tasks or bimanual tasks with one hand. However, recent research has introduced the use of two haptic devices for bimanual assembly. We propose a more natural and low-cost bimanual interaction than existing ones based on Markerless motion capture (Mocap) systems. Specifically, this paper presents two interactions based on a Markerless Mocap technology and one interaction based on combining Markerless Mocap technology with haptic technology. A set of experiments following a within-subjects design have been implemented to test the usability of the proposed interfaces. The Markerless Mocap-based interactions were validated with respect to two-haptic-based interactions, as the latter has been successfully integrated into bimanual assembly simulators. The pure Markerless Mocap interaction proved to be either the most or least efficient depending on the configuration (with 2D or 3D tracking, respectively). Usability results among the proposed interactions and the two-haptic based interaction showed no significant differences. These results suggest that Markerless Mocap or hybrid interactions are valid solutions for simulating bimanual assembly tasks when the precision of the motion is not critical. The decision on which technology to use should depend on the trade-off between the precision requested to simulate the task, the cost, and inner features of the technology.

Seeing Movement: On Motion Capture Animation and James Cameron’sAvatar

This article proposes that motion capture (mocap) animation relates movement to cinema in a unique way, in that rather than being a quality of the profilmic, in mocap animation movement is itself directly the profilmic. Motion capture records imagery consisting of data of a profilmic object’s positional change in space, rather than data of the object itself. Using this critical distinction between movement and object, the author argues that the experience of mocap changes the nature of the image so that it involves, or is, a specific sense of being, rather than seeing. Due to its thematic treatment of seeing as well as its own application of mocap technology, she also draws on James Cameron’s film Avatar (2009) as an illustration for this thesis on mocap and seeing/being. In the process, she revisits our experiences of seeing light when watching films and considers how mocap and the experience of movement change our engagement with cinema. This discussion is thus not only about our understanding of and interaction with the moving image, but also points to how we can understand movement and being, and the sum of our sensory experiences in the world of cinema ensconced in light and darkness.