Convolutional network architectures for super-resolution/sub-pixel mapping of drone-derived images

Abstract

Spatial resolution enhancement is a pre-requisite for integrating unmanned aerial vehicle (UAV) datasets with the data from other sources. However, the mobility of UAV platforms, along with radiometric and atmospheric distortions, makes the task difficult. In this paper, various convolutional neural network (CNN) architectures are explored for resolving the issues related to sub-pixel classification and super-resolution of drone-derived datasets. The main contributions of this work are: 1) network-inversion based architectures for super-resolution and sub-pixel mapping of drone-derived images taking into account their spectral-spatial characteristics and the distortions prevalent in them 2) a feature-guided transformation for regularizing the inversion problem 3) loss functions for improving the spectral fidelity and inter-label compatibility of coarser to finer-scale mapping 4) use of multi-size kernel units for avoiding over-fitting. The proposed approach is the first of its kind in using neural network inversion for super-resolution and sub-pixel mapping. Experiments indicate that the proposed super-resolution approach gives better results in comparison with the sparse-code based approaches which generally result in corrupted dictionaries and sparse codes for multispectral aerial images. Also, the proposed use of neural network inversion, for projecting spatial affinities to sub-pixel maps, facilitates the consideration of coarser-scale texture and color information in modeling the finer-scale spatial-correlation. The simultaneous consideration of spectral bands, as proposed in this study, gives better super-resolution results when compared to the individual band enhancements. The proposed use of different data-augmentation strategies, for emulating the distortions, improves the generalization capability of the framework. Sensitivity of the proposed super-resolution and sub-pixel mapping frameworks with regard to the network parameters is thoroughly analyzed. The experiments over various standard datasets as well as those collected from known locations indicate that the proposed frameworks perform better when compared to the prominent published approaches.