Deep locally linear embedding network

Abstract

Recent years have seen great progress in many deep learning-based super-resolution methods that learn global responses through the non-local strategy. However, improper linear fitting tends to produce unsatisfactory results with notable artifacts. To relieve the improper linear fitting and reuse the hierarchical features, we propose an end-to-end locally linear embedding super-resolution network, termed as LLE-Net, based on the assumption that the local geometric relationship in the low-resolution manifold space also exists in high-level feature manifold space. The proposed LLE-Net includes two major components: 1) the local linear embedding block (LLEB) and 2) the hierarchical non-local block (HNB). In particular, LLEB searches sparse and similar feature maps and embeds the geometric relationship into the high-level feature space, allowing more attention to be paid to objects’ textural details. To promote the representation ability, the proposed HNB is able to explore layer- and pixel-level interdependencies. We conduct extensive experiments to evaluate the superiority of LLE-Net via two groups of experiments: (1) super-resolution tasks on two satellite image datasets and a satellite video image dataset, and (2) the subsequent high-level image processing tasks (i.e., satellite semantic segmentation). The proposed method achieves an 0.7 dB improvement in PSNR value on the Draper dataset and 0.07% segmentation accuracy improvement. The source code of the proposed LLE-Net is publicly available.