Fusion and super-resolution in multispectral data using neural networks for improved RGB representation

Abstract

A new method, which aims at creating a high-quality RGB fused representation with increased spatial resolution from multispectral data, is proposed. For this purpose, a feed-forward neural network is employed, and a successive training procedure is applied in order to incorporate into the network structure knowledge about recovering folded frequencies and, therefore, obtain output colour images with fine resolution. The successive training of the network takes place in three consecutive steps. The training data used in each step are of higher resolution than those used in the previous step. A decimation procedure, based on a generally acceptable super-resolution model, is employed to create from the original multispectral bands all the required down-sampled series of input training data vectors. The output (target) training vectors are obtained by fusing the multispectral images. Fusion is carried out by means of segmented principal component transformation in order to adjust the type of spectral characteristics transferred to the final RGB image components. MERIS multispectral data from the ENVISAT satellite are employed to demonstrate the performance of the proposed method. The method can be applied to obtaining high-resolution multispectral data representation, suitable for monitoring and inspection by humans. Furthermore, it can be applied without substantial modification to other types of multi-modal data, such as medical images, for yielding improved RGB representation to specialists.