Event-Based Semantic Image Adaptation for User-Centric Mobile Display Devices

Abstract

This paper proposes a semantic image adaptation scheme for heterogeneous mobile display devices. This scheme aims to provide mobile users with the most desired image content by integrating the content semantic importance with user preferences under limited mobile display constraints. The main contributions of the proposed scheme are: 1) seamless integration of mobile user supplied query information with low level image features to identify semantically important image contents; 2) integration of semantic importance and user feedback to dynamically update mobile user preferences; and 3) perceptually optimized adaptation for image display on mobile devices. In order to bridge the semantic gap for adaptation, we design a Bayesian fusion approach to properly integrate low level features with high level semantics. To accommodate the variation of user preferences, the system involves mobile users in the adaptation process with only a few simple feedbacks so as to present to the users most interesting content on mobile devices. Eventually, perceptually optimized adaptation is performed to present the best image content for mobile users according to mobile display capacities. Extensive experiments have been carried out based on several common events defined in Kodak's consumer image database. These experiments show that by utilizing the proposed semantic adaptation scheme with integration of the semantics and mobile user preferences, perceptually relevant adaptation can be effectively carried out to tailor the image towards user intentions under the mobile environment constraints.