An enhanced personal photo recommendation system by fusing contextual and textual features on mobile device

Abstract

As a main means to record scene in personal daily life, personal photos convey high-level semantic information (e.g., who, what, when, where) of an activity user engaged in. Different from other information retrieval tasks, personal photo recommendation depends on the measure of activity relevancy which is implicitly embedded in photos. Spurred by this observation, an enhanced recommendation approach by fusing both contextual and textual features is proposed. First, contextual relevancy is incrementally refined with an enhanced temporal and spatial clustering method respectively. Second, textual similarity of photo annotations is calculated using WordNet to augment the activity relevancy. Third, a fuzzy decision based multi-criteria ranking algorithm i.e., Preference Ranking Organization Method of Enrichment Evaluation (PROMETHEE) is adopted to make recommendations when giving an entry photo. A prototype has been developed on mobile device to illustrate this concept. Experiment results on a real dataset which contains 10,827 photos collected from 50 volunteers during 12 months demonstrate that our approach is more accurate than traditional schemes.