Toward Content-Driven Intelligent Authoring of Mulsemedia Applications

Abstract

Synchronization of sensory effects with multimedia content is a nontrivial and error-prone task that can discourage authoring of mulsemedia applications. Although there are authoring tools that perform some automatic authoring of sensory effect metadata, the analysis techniques that they use are not generally enough to identify complex components that may be related to sensory effects. In this article, we present a new method, which allows the semiautomatic definition of sensory effects in an authoring tool. We outline a software component to be integrated into authoring tools that uses content analysis assistance to indicate moments of sensory effects activation, according to author preferences. The proposed method was implemented in the STEVE 2.0 authoring tool and an evaluation was performed to assess the precision of the generated sensory effects in comparison with human authoring. This solution is expected to considerably reduce the effort of synchronizing audiovisual content with sensory effects—in particular, by easing the author’s repetitive task of synchronizing recurring effects with lengthy media.