LabelMovie: Semi-supervised machine annotation tool with quality assurance and crowd-sourcing options for videos

Abstract

For multiple reasons, the automatic annotation of video recordings is challenging. The amount of database video instances to be annotated is huge, tedious manual labeling sessions are required, the multi-modal annotation needs exact information of space, time, and context, and the different labeling opportunities require special agreements between annotators, and alike. Crowd-sourcing with quality assurance by experts may come to the rescue here. We have developed a special tool: individual experts can annotate videos over the Internet, their work can be joined and filtered, the annotated material can be evaluated by machine learning methods, and automated annotation may start according to a predefined confidence level. A relatively small number of manually labeled instances may efficiently bootstrap the machine annotation procedure. We present the new mathematical concepts and algorithms for semi- supervised induction and the corresponding manual annotation tool which features special visualization methods for crowd- sourced users. A special feature is that the annotation tool is usable for users not familiar with machine learning methods; for example, we allow them to ignite and handle a complex bootstrapping process. I. INTRODUCTION Annotation of videos is of great interest for content providers for monitoring, surveillance, meteorology, maritime processes and similar authoring tasks. Advanced solutions should include more precise content-based annotations by extending common data annotation tools by the (semi-) auto- matic annotation of video recordings. The annotations them- selves serve video retrieval and browsing. Annotation can be guided by contextual and content-based information, and can rely on auditory, visual, textual, color information. Annotation can aim at more sophisticated goals, such as the annotation of player's behavior in an educational game and thus help in the personalization of educational training material. It can also assist authoring, for example. In fact, methods for media annotation to perform the whole application cycle of