Content-aware Retargeting Methods Research Articles

Pyramidal modeling of geometric distortions for retargeted image quality evaluation

Content-aware retargeting methods are used to adjust images to different resolutions and aspect ratios with low deformation and information loss in salient regions. Effective objective quality assessment of retargeted images can provide a way to improve retargeting methods. The non-uniform geometrical degradations caused by retargeting algorithms make it impossible to use traditional image quality assessment metrics for retargeted images. Although some quality evaluation methods have been proposed till now, the resulted quality scores are not well correlated with the subjective ones. In this paper we propose a pyramidal global-to-local pooling method to combine pixel/block deformation measures. In each level of locality, the Euclidean distance between the retargeted and original image is used as an individual feature. Therefore, in addition to the global summation, assessment of local deformations, contributes toward better quality evaluation. Learning a regression model based on the extracted features results in better performance compared to relevant existing retargeted image quality methods.

Video retargeting using optimized crop-and-scale

Mobile devices are increasingly powerful in multimedia transmitting and browsing. However, the small screen and different aspect ratios of mobile devices lower visual quality while watching images or videos. Video retargeting is aim at fitting an existing video into arbitrary size and aspect ratio. Previous content-aware retargeting methods are mostly high computational cost, which limits their applications on the portable devices with low computation ability. In this paper, a new crop-and-scale approach is presented to adapt video to better suit the target display. We automatically find the optimal parameters of cropping window using the dynamic programming method, and then scale it to fit the target display. The cropping window can smoothly shift during a shot to follow the movement of important objects. The retargeting results using our approach introduce no deformation and jitter effects over the whole video. Experimental results show the success of our approach on adapting a variety of source videos to small display sizes.

Content-Aware Retargeting For Soccer Video Adaptation

Abstract A content-aware retargeting method is proposed for adapting soccer video to heterogeneous terminals. According to domain-specific knowledge, ball, player and player’s face are defined as user interested objects (UIOs) in different view-types. The UIOs are extracted by semantic analysis on soccer video, and then a region of interest (ROI) of each shot is determined jointly by three factors: terminal size, scaling factor and aspect ratio. The proposed method optimizes the retargeted region to contain more semantic content while adapting the constraint of terminal screen. The simulation results prove that the proposed CAR system wins better viewing experiences than the traditional methods such as resizing in a “Letter box” mechanism or cropping directly.

Motion-based video retargeting with optimized crop-and-warp

We introduce a video retargeting method that achieves high-quality resizing to arbitrary aspect ratios for complex videos containing diverse camera and dynamic motions. Previous content-aware retargeting methods mostly concentrated on spatial considerations, attempting to preserve the shape of salient objects in each frame by removing or distorting homogeneous background content. However, sacrificeable space is fundamentally limited in video, since object motion makes foreground and background regions correlated, causing waving and squeezing artifacts. We solve the retargeting problem by explicitly employing motion information and by distributing distortion in both spatial and temporal dimensions. We combine novel cropping and warping operators, where the cropping removes temporally-recurring contents and the warping utilizes available homogeneous regions to mask deformations while preserving motion. Variational optimization allows to find the best balance between the two operations, enabling retargeting of challenging videos with complex motions, numerous prominent objects and arbitrary depth variability. Our method compares favorably with state-of-the-art retargeting systems, as demonstrated in the examples and widely supported by the conducted user study.

Motion-based video retargeting with optimized crop-and-warp

We introduce a video retargeting method that achieves high-quality resizing to arbitrary aspect ratios for complex videos containing diverse camera and dynamic motions. Previous content-aware retargeting methods mostly concentrated on spatial considerations, attempting to preserve the shape of salient objects in each frame by removing or distorting homogeneous background content. However, sacrificeable space is fundamentally limited in video, since object motion makes foreground and background regions correlated, causing waving and squeezing artifacts. We solve the retargeting problem by explicitly employing motion information and by distributing distortion in both spatial and temporal dimensions. We combine novel cropping and warping operators, where the cropping removes temporally-recurring contents and the warping utilizes available homogeneous regions to mask deformations while preserving motion. Variational optimization allows to find the best balance between the two operations, enabling retargeting of challenging videos with complex motions, numerous prominent objects and arbitrary depth variability. Our method compares favorably with state-of-the-art retargeting systems, as demonstrated in the examples and widely supported by the conducted user study.