Personalized video adaptation framework (PIAF): high-level semantic adaptation

Abstract

Despite much work on Universal Multimedia Experience (UME), existing video adaptation approaches cannot yet be considered as truly user-centric, mostly due to their poor handling of semantic user preferences. Indeed, these works mainly concentrate on lower-level user preferences but do neither consider any fine-grained object-level adaptation nor evaluate different adaptation options based on predicted user expectations. Moreover, these works do not provide owners with property rights that enable them to place restrictions on the types of modifications to be made to the video content. To address these shortcomings, we propose the Personalized vIdeo Adaptation Framework (PIAF) for high-level semantic video adaptation. PIAF is a fully integrated framework providing all the requirements for a semantic video adaptation. It defines a video annotation model and a user profile model comprising semantic constraints that are delineated in a consistent way, based on the standards MPEG-7 and MPEG-21. At the heart of the framework, the Adaptation Decision Taking Engine (ADTE) computes utility values for different adaptation options, considering each shot separately. The corresponding utility function evaluates the possible choices by evaluating multiple parameters that capture different dimensions of a multimedia experience: amount of modified content, modifications to key objects and shots with respect to the semantic integrity of the original content, expected processing cost of the adaptation, and the anticipated visual and temporal quality of the adapted content. Furthermore, the ADTE can deal with intellectual property issues by selecting an adaptation plan of good quality that also satisfies constraints specified by the content owner. This paper places a significant emphasis on theoretical details of the utility function and the computation of the adaptation plan. It also presents the results and evaluation of the adaptation process both in simulation and user study.