Active capture: integrating human-computer interaction and computer vision/audition to automate media capture

Abstract

While the devices for media capture have advanced from mechanical to computational since the invention of photography and motion pictures in the 19th century, their underlying user interaction paradigms have remained largely unchanged. Current interaction techniques for media capture do not leverage computation to solve key problems: the skill required to capture high quality media assets; the effort required to select useable assets from captured assets; and the lack of metadata describing the content and structure of media assets that could enable them to be retrieved and (re)used. We describe a new interaction and processing paradigm for media capture that redefines capture as a control process with feedback. By integrating human-computer interaction and computer vision and audition into an "active capture" process, we overcome the limitations of current media capture devices, algorithms, and interaction techniques. Active capture leverages media production knowledge to automate direction and cinematography and thus enables the automated production of annotated, high quality, reusable media assets.