Semantic indexing and retrieval of home photos

Abstract

With rapid advances in sensor, storage, processor, and communication technologies, consumers can now afford to create, store, process, and share large digital photo collections. With more and more digital photos accumulated, consumers need effective and efficient tools to organize and access photos in a semantically meaningful way without too much manual annotation effort. From user studies, we confirm that users prefer to organize and access photos along semantic axes such as event, people, time, and place. As a matter of fact, research on content-based image retrieval in the last decade is yet to bridge the semantic gap between feature-based indexes computed automatically and human preferences on query and retrieval. In this paper, we attempt to address this semantic gap by focusing on the notion of in home photos. First we propose event taxonomy for home photos. Next we propose a computational learning framework to construct event models from sample photos with event labels given by a user and to compute relevance measures of unlabeled photos to the event models, Last but not least, we demonstrate event-based retrieval on 2400 genuine home photos using our proposed approach.