Abstract
Image and video processing based on geometric principles typically changes the rectangular shape of video frames to an irregular shape. This paper presents a warping based approach for rectangling such irregular frame boundaries in space and time, i.e., making them rectangular again. To reduce geometric distortion in the rectangling process, we employ content-preserving deformation of a mesh grid with line structures as constraints to warp the frames. To conform to the original inter-frame motion, we keep feature trajectory distribution as constraints during motion compensation to ensure stability after warping the frames. Such spatially and temporally optimized warps enable the output of regular rectangular boundaries for the video frames with low geometric distortion and jitter. Our experiments demonstrate that our approach can generate plausible video rectangling results in a variety of applications.
Highlights
In recent years, geometry techniques have been widely used in image and video processing, such as image resizing or retargetting [1], perspective editing [2, 3], video stabilization [4], and panoramic stitching [5,6,7,8,9]
Unlike traditional image and video processing based on pixel operations, geometry-driven methods typically adopt a mesh grid structure in the image plane and manipulate the grid points to drive the processing of the enclosed patches, enabling more flexible and coherent processing of the image and video content [10]
Manuscript received: 2021-01-11; accepted: 2021-03-01 desired balance between efficiency and effectiveness, but they usually result in warping of the boundary of images or video frames when directly operating on grid points, changing the rectangular shape to an irregular boundary
Summary
Geometry techniques have been widely used in image and video processing, such as image resizing or retargetting [1], perspective editing [2, 3], video stabilization [4], and panoramic stitching [5,6,7,8,9]. Unlike traditional image and video processing based on pixel operations, geometry-driven methods typically adopt a mesh grid structure in the image plane and manipulate the grid points to drive the processing of the enclosed patches, enabling more flexible and coherent processing of the image and video content [10] These methods achieve the Manuscript received: 2021-01-11; accepted: 2021-03-01 desired balance between efficiency and effectiveness, but they usually result in warping of the boundary of images or video frames when directly operating on grid points, changing the rectangular shape to an irregular boundary. Because most display screens are rectangular, it is necessary to make images and videos with irregular boundaries rectangular for normal display on common screens This paper mainly addresses the problem of rectangling video frames with irregular boundaries
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
