Investigation and evaluation of algorithms for unmanned aerial vehicle multispectral image registration

Abstract

Automatically registration of unmanned aerial vehicle (UAV) multispectral images is fundamental for subsequent applications. Although many studies exist for visible camera images and satellite multispectral image registration, studies for UAV are still rare. Under this context, this study firstly evaluates the performance of several widely used traditional methods (i.e., SIFT, SURF, ORB, and CFOG) for visible camera and satellite images in UAV multispectral image registration. This study further proposes an unsupervised and end-to-end deep learning network (i.e., DSIM) for multispectral image registration. An evident feature of DSIM is to regress the homography parameters with convolutional neural networks and to use the pyramid structural similarity loss to optimize the network. 1200 groups of UAV multispectral images acquired over three different sites in four months are used to comprehensively test the aforementioned five methods. Results show that CFOG achieves the highest correct matching rate in the test set, followed by DSIM and SIFT. Nevertheless, DSIM is more robust in images with weak or repeated texture than CFOG and SIFT. In addition, performance of CFOG and SIFT is closely related to the number of the found matching points. Based on the findings, we propose a multi-method ensemble strategy to combine CFOG, DSIM, and SIFT according to the number of the found matching points. This strategy outperforms the individual methods with a correct matching rate of 96.2%. Lower correct matching rate of CFOG + SIFT confirms that DSIM and traditional methods are very complementary in UAV multispectral image registrations.