Spatiotemporal image fusion using multiscale attention-aware two-stream convolutional neural networks

Abstract

Spatiotemporal fusion (STF) is considered as a promising way to produce remote sensing images at fine scales in both space and time by blending two types of satellite images. The learning-based STF approaches with deep convolutional neural networks can provide the unified framework to address both gradual and abrupt changes. This paper intends to develop an enhanced learning-based STF using multiscale attention-aware two-stream convolutional neural networks (MACNN). With a coarse image at the prediction date and two pairs of coarse and fine images at other dates as inputs, it employs a multiscale module to characterize different sizes of objects and a spatial and channel attention module to emphasize important information in feature learning. Two experiments on real Landsat and MODIS images are conducted to demonstrate the effectiveness of the proposed MACNN and it outperforms four existing STF methods in both visual and quantitative.