Enhancing Person Annotation for Personal Photo Management Applications

Abstract

This paper addresses a sub-problem of the broad annotation problem, namely "person annotation", associated with personal digital photo management and investigates approaches to enhancing person annotation in personal photo management applications. We study a number of approaches to enhance the performance of semi-automatic person annotation using real-life personal photo collections as the test data. To this end, face and body-patch features are employed to describe the appearance of a person as a means to more effectively capture the identities of re-appearing people in personal photo archives. Experiments are carried out to identify a suitable initial annotation method, compare the performances of event-constrained person matching with global person matching, and the effect of the size of initial annotation on the overall performance of person annotation in real-life personal photo archives. The evaluation results, presented in terms of H-hit rate figures, illustrate that using event-constrained person matching with event-based initial annotation proves to be a better performing approach than global person matching for person annotation in personal photo archives. Results also clearly demonstrate the nature of compromise that needs to be made when annotating large photo collections in terms of accuracy against user-interaction.