A novel camera path planning algorithm for real-time video stabilization

Yun Gu Lee

doi:10.1186/s13640-019-0463-2

Abstract

In video stabilization, a steady camera path plan is as important as accurate camera motion prediction. While several camera path planning algorithms have been studied, most algorithms are used for post-processing video stabilization. Since all of the frames are accessible in post-processing video stabilization, a camera path planning method can generate a good camera path, considering the camera movements at all of the frames. Meanwhile, the number of accessible frames in real-time video stabilization is limited. Thus, smooth camera path planning is a challenging issue in real-time video stabilization. For example, a camera path planner does not know in advance the locations of sudden camera motions, so it is not easy to compensate a sudden camera movement. Therefore, this paper proposes a novel camera path planning algorithm for real-time video stabilization. A camera path planning method should fully utilize the given image margin to provide steady camera paths. In contrast, it should retain the certain amount of an unused image margin for use in frames with dynamic or sudden camera movements. To resolve this problem, the proposed algorithm uses two terms related to the steady camera path and the amount of image margin, and it cross-optimizes the two terms to provide a new camera path on-the-fly. Experimental results show that the proposed algorithm provides excellent performance in real-time video stabilization while requiring negligible computation.

Highlights

With the recent advancement in camera technology, the shooting and sharing of videos with consumer cameras has significantly increased
In this paper, we propose a new camera path planning algorithm for real-time video stabilization by cross-optimizing two terms related to steady camera path and the amount of image margin
5 Conclusion This paper presents a novel camera path planning algorithm for real-time video stabilization by cross-optimizing two terms related to steady camera path and the amount of image margin

Summary

Introduction

With the recent advancement in camera technology, the shooting and sharing of videos with consumer cameras has significantly increased. Motion-inpainting does not provide satisfactory performance in some cases and has a high computational cost Another approach is to forcibly modify the new camera path to fit the original frame, but compulsory modification of the new camera path will degrade the visual quality of the stabilized videos. The size of output videos can be set so that the cropping window always fits the original frame While this method significantly reduces the field of view (or the size of output views) for videos having dynamic camera motions, it can maximize dramatic cinematographic effect [34]. In this paper, we propose a new camera path planning algorithm for real-time video stabilization by cross-optimizing two terms related to steady camera path and the amount of image margin. One term renders the new camera trajectory smooth, and the other term retains the certain amount of an unused image margin for use in dynamic or sudden camera motions

Background

Method—camera path planning

Method

Conclusion