Accelerated RANSAC for Accurate Image Registration in Aerial Video Surveillance

Jin Zheng,Wei Peng,Yue Wang,Bo Zhai

doi:10.1109/access.2021.3061818

Abstract

Compared with ground views and direct overhead views (for orbital satellites), aerial robotics allow for capturing videos from diverse viewpoints and scenes, thus, the content of aerial image is complex and changeable, and aerial video has complex inter-frame transforms stemming from the blend of camera motion, platform motion and jitter. In addition, poor quality and similar texture are common in long-distance and large-scale aerial video surveillance. All of these interferences make image registration of aerial video difficult. This article puts forward one image registration method suited to aerial video on the basis of the hypothesize-and-verify of RANSAC. The proposed accelerated RANSAC, named PSSC-RANSAC (Prior Sampling & Sample Check RANSAC), incorporates prior sampling, which comes from three levels of sample evaluation, including texture magnitude, spatial consistency and feature similarity, to generate more possibly correct samples in priority. Furthermore, prior information of sample, quality of sample subset and subset invariability are together used to check the sample subsets, and the incompatible arrangements of subsets are immediately ruled out in sample check stage, which speeds up the iteration further. Results of the experiment have proved the good performance of the presented PSSC-RANSAC at 90% contamination level. For typical image pairs, the number of iterations is reduced by at least 16.67% and evaluation computation is reduced by at least 11.01% compared with SVH-RANSAC, and the re-projection error is decreased by at least 4.44% and 6.31% compared with RANSAC and SVH-RANSAC, respectively. It can overcome the interferences, and is very suitable for image registration of aerial images.

Highlights

With the continuous and rapid development of aerial vehicle technology, aerial video surveillance has been widely used, which involves many tasks in military and civil monitoring
For this kind of moving platform of aerial vehicle, image registration is usually adapted to compensate the background motion with a 2D parametric transformation [2], which includes the motion of platform and camera, as well as jitter
(3) Besides prior sampling, PSSC-RANSAC further combines sample subset quality and subset invariability together to check the sample subsets. This operation considers local information around feature point, spatial relationship of sample subset and invariability of subset correspondences, and carries out sample check quickly, which speeds up the iteration further

Summary

INTRODUCTION

With the continuous and rapid development of aerial vehicle technology, aerial video surveillance has been widely used, which involves many tasks in military and civil monitoring. The majority of existing methods adopt ‘random sample consensus’ to capture the correct feature matches, and utilize local image information, including pixel intensity, color and texture, near feature points to evaluate the similarity of correspondences. In order to verify the influence of the above three measures on the number of feature points, three pairs of images whose differences are mainly reflected in contrast, coarseness and roughness, and the results of extracted feature points are shown in Figure. and Table 1, respectively. Coarseness and roughness all affect the number of feature points, these measures are combined to define a new metric, which is texture magnitude. The above tests illustrated in Figure. have shown that the feature points distributed in the region with high texture magnitude have higher location accuracy They share a higher fraction of inliers. Such simple check is cheaper, especially than that costly model generation stage and verification stage

FRAMEWORK OF THE PROPOSED METHOD

Findings

CONCLUSION