Real-Time Video Stitching Using Camera Path Estimation and Homography Refinement

Jaeyoung Yoon,Daeho Lee

doi:10.3390/sym10010004

Abstract

We propose a novel real-time video stitching method using camera path estimation and homography refinement. The method can stably stitch multiple frames acquired from moving cameras in real time. In the proposed method, one initial between-camera (BC) homography and each camera path (CP) homography are used to estimate the BC homography at every frame. The BC homography is refined by using block matching to adjust the errors of estimated CPs (homography refinement). For fast processing, we extract features using the difference of intensities and use the optical flow to estimate camera motion (CM) homographies, which are multiplied with the previous CMs to calculate CPs (camera path estimations). In experiments, we demonstrated the performance of the CP estimation and homography refinement approach by comparing it with other methods. The experimental results show that the proposed method can stably stitch two image sequences at a rate exceeding 13 fps (frames per second).

Highlights

Video stitching is a technique that stitches multiple videos acquired from moving cameras.By allowing the videos to have a wider field of view (FOV) and higher a resolution, users can be immersed in the video
To evaluate the performance of the proposed method, we performed three experiments as follows: (1) we evaluated the performance of our method using a variety of samples, (2) we demonstrated the performance of this camera path (CP) estimation and homography refinement approach by comparing it with other methods, and (3) we found the optimal parameters
We propose a novel video stitching method that combines multiple input videos from handheld cameras in real-time

Summary

Introduction

Video stitching is a technique that stitches multiple videos acquired from moving cameras. By allowing the videos to have a wider field of view (FOV) and higher a resolution, users can be immersed in the video. Video stitching has been applied in various fields, including surveillance systems [1,2,3], teleconferencing, immersive virtual reality, and sports broadcasting. Most video stitching methods are for cameras mounted on fixed rigs [4]. The use of fixed rigs is advantageous in that the processing time is short by using a fixed homography. It is difficult for the general public to utilize these approaches due to the expensive camera systems

Objectives

Methods

Results

Discussion

Conclusion