Remote Sensing Image Reconstruction Research Articles

Target detection algorithm based on super- resolution color remote sensing image reconstruction

An improved generative adversarial network model is adopted to improve the resolution of remote sensing images and the target detection algorithm for color remote sensing images. The main objective is to solve the problem of training super-resolution reconstruction algorithms and missing details in reconstructed images, aiming to achieve high-precision detection of medium and low-resolution color remote sensing targets. First, a lightweight image super-resolution reconstruction algorithm based on an improved generative adversarial network (GAN) is proposed. This algorithm combines the pixel attention mechanism and up-sampling method to restore image details. It further integrates edge-oriented convolution modules into traditional convolution to reduce model parameters and achieve better feature collection. Then, to further enhance the feature collection ability of the model, the YOLOv4 object detection algorithm is also improved. This is achieved by introducing the Focus structure into the backbone feature extraction network and integrating multi-layer separable convolutions to improve the feature extraction ability. The experimental results show that the improved target detection algorithm based on super resolution has a good detection effect on remote sensing image targets. It can effectively improve the detection accuracy of remote sensing images, and have a certain reference significance for the realization of small target detection in remote sensing images.

A Super-Resolution Network for High-Resolution Reconstruction of Landslide Main Bodies in Remote Sensing Imagery Using Coordinated Attention Mechanisms and Deep Residual Blocks

The lack of high-resolution training sets for intelligent landslide recognition using high-resolution remote sensing images is a major challenge. To address this issue, this paper proposes a method for reconstructing low-resolution landslide remote sensing images based on a Super-Resolution Generative Adversarial Network (SRGAN) to fully utilize low-resolution images in the process of constructing high-resolution landslide training sets. First, this paper introduces a novel Enhanced Depth Residual Block called EDCA, which delivers stable performance compared to other models while only slightly increasing model parameters. Secondly, it incorporates coordinated attention and redesigns the feature extraction module of the network, thus boosting the learning ability of image features and the expression of high-frequency information. Finally, a residual stacking-based landslide remote sensing image reconstruction strategy was proposed using EDCA residual blocks. This strategy employs residual learning to enhance the reconstruction performance of landslide images and introduces LPIPS for evaluating the test images. The experiment was conducted using landslide data collected by drones in the field. The results show that compared with traditional interpolation algorithms and classic deep learning reconstruction algorithms, this approach performs better in terms of SSIM, PSNR, and LPIPS. Moreover, the network can effectively handle complex features in landslide scenes, which is beneficial for subsequent target recognition and disaster monitoring.

A lightweight distillation CNN-transformer architecture for remote sensing image super-resolution

ABSTRACT Remote sensing images exhibit rich texture features and strong autocorrelation. Although the super-resolution (SR) method of remote sensing images based on convolutional neural networks (CNN) can capture rich local information, the limited perceptual field prevents it from establishing long-distance dependence on global information, leading to the low accuracy of remote sensing image reconstruction. Furthermore, it is difficult for existing SR methods to be deployed in mobile devices due to their large network parameters and high computational demand. In this study, we propose a lightweight distillation CNN-Transformer SR architecture, named DCTA, for remote sensing SR, addressing the aforementioned issues. Specifically, the proposed DCTA first extracts the coarse features through the coarse feature extraction layer and then learns the deep features of remote sensing at different scales by fusing the feature distillation extraction module of CNN and Transformer. In addition, we introduce the feature fusion module at the end of the feature distillation extraction module to control the information propagation, aiming to select the informative components for better feature fusion. The extracted low-resolution (LR) feature maps are reorganized through the up-sampling module to obtain high-resolution (HR) feature maps with high accuracy to generate high-quality HR remote sensing images. The experiments comparing different methods demonstrate that the proposed approach performs well on multiple datasets, including NWPU-RESISC45, Draper, and UC Merced. This is achieved by balancing reconstruction performance and network complexity, resulting in both competitive subjective and objective results.

Adaptive-Attention Completing Network for Remote Sensing Image

The reconstruction of missing pixels is essential for remote sensing images, as they often suffer from problems such as covering, dead pixels, and scan line corrector (SLC)-off. Image inpainting techniques can solve these problems, as they can generate realistic content for the unknown regions of an image based on the known regions. Recently, convolutional neural network (CNN)-based inpainting methods have integrated the attention mechanism to improve inpainting performance, as they can capture long-range dependencies and adapt to inputs in a flexible manner. However, to obtain the attention map for each feature, they compute the similarities between the feature and the entire feature map, which may introduce noise from irrelevant features. To address this problem, we propose a novel adaptive attention (Ada-attention) that uses an offset position subnet to adaptively select the most relevant keys and values based on self-attention. This enables the attention to be focused on essential features and model more informative dependencies on the global range. Ada-attention first employs an offset subnet to predict offset position maps on the query feature map; then, it samples the most relevant features from the input feature map based on the offset position; next, it computes key and value maps for self-attention using the sampled features; finally, using the query, key and value maps, the self-attention outputs the reconstructed feature map. Based on Ada-attention, we customized a u-shaped adaptive-attention completing network (AACNet) to reconstruct missing regions. Experimental results on several digital remote sensing and natural image datasets, using two image inpainting models and two remote sensing image reconstruction approaches, demonstrate that the proposed AACNet achieves a good quantitative performance and good visual restoration results with regard to object integrity, texture/edge detail, and structural consistency. Ablation studies indicate that Ada-attention outperforms self-attention in terms of PSNR by 0.66%, SSIM by 0.74%, and MAE by 3.9%, and can focus on valuable global features using the adaptive offset subnet. Additionally, our approach has also been successfully applied to remove real clouds in remote sensing images, generating credible content for cloudy regions.

Thick Cloud Removal in Multi-Temporal Remote Sensing Images via Frequency Spectrum-Modulated Tensor Completion

Clouds often contaminate remote sensing images, which leads to missing land feature information and subsequent application degradation. Low-rank tensor completion has shown great potential in the reconstruction of multi-temporal remote sensing images. However, existing methods ignore different low-rank properties in the spatial and temporal dimensions, such that they cannot utilize spatial and temporal information adequately. In this paper, we propose a new frequency spectrum-modulated tensor completion method (FMTC). First, remote sensing images are rearranged as third-order spatial–temporal tensors for each band. Then, Fourier transform (FT) is introduced in the temporal dimension of the rearranged tensor to generate a spatial–frequential tensor. In view of the fact that land features represent low-frequency components and fickle clouds represent high-frequency components in the time domain, we chose adaptive weights for the completion of different low-rank spatial matrixes, according to the frequency spectrum. Then, Invert Fourier Transform (IFT) was implemented. Through this method, the joint low-rank spatial–temporal constraint was achieved. The simulated data experiments demonstrate that FMTC is applicable on different land-cover types and different missing sizes. With real data experiments, we have validated the effectiveness and stability of FMTC for time-series remote sensing image reconstruction. Compared with other algorithms, the performance of FMTC is better in quantitative and qualitative terms, especially when considering the spectral accuracy and temporal continuity.

Remote Sensing Image Reconstruction Based on Improved Adaptive Tensor Nuclear Norm Minimization

Incomplete remote sensing image observation data require proper processing to restore the original data. Nevertheless, the lack of information, due to noise and dead pixels, along with the quality of the acquired images that may be severely degraded, limits the accuracy of later processing. Therefore, this paper proposes a remote sensing image restoration scheme that employs the low-rank structure of the Bayesian probability tensor latent space and a new tensor completion method, which has strong robustness for the model selection process of the following remote sensing image data analysis. Considering the data space, we incorporate an adaptive nuclear norm regularization into the processing of latent factors that performs singular value decomposition on a more accurate scale. The proposed model adopts two algorithms: Augmented Lagrange Multiplier (ALM) and Alternating Least Squares (ALS). Several synthetic and real-data-based experiments illustrate the inherent ability of our method in restoring original information and preventing overfitting problems, even when a vast number of items are missing. In addition, the combination of adaptive nuclear norm regularization and tensor decomposition is stable, settling a high sensitivity of rank selection and low computational efficiency in traditional tensor reconstruction methods.

Remote sensing image reconstruction using an asymmetric multi-scale super-resolution network

Remote sensing image reconstruction using an asymmetric multi-scale super-resolution network

Superresolution Reconstruction of Remote Sensing Image Based on Generative Adversarial Network

To recreate high-resolution, more detailed remote sensing images from existing low-resolution photos, this technique is known as remote sensing image superresolution reconstruction, and it has numerous uses. As an important research hotspot of neural networks, generative adversarial network (GAN) has made outstanding progress for image superresolution reconstruction. It solves the computational complexity and low reconstructed image quality of standard superresolution reconstruction algorithms. This research offers a superresolution reconstruction strategy with a self-attention generative adversarial network to improve the quality of reconstructed superresolution remote sensing images. The self-attention strategy as well as residual module is utilized to build a generator in this model that transforms low-resolution remote sensing images into superresolution ones. It aims to determine the discrepancy between a reconstructed picture and a true picture by using a deep convolutional network as a discriminator. For the purpose of enhancing the accuracy, content loss is used. This is done to obtain accurate detail reconstruction. According to the findings of the experiments, this approach is capable of regenerating higher-quality images.

Superresolution Reconstruction of Remote Sensing Image Based on Middle-Level Supervised Convolutional Neural Network

Image has become one of the important carriers of visual information because of its large amount of information, easy to spread and store, and strong sense of sense. At the same time, the quality of image is also related to the completeness and accuracy of information transmission. This research mainly discusses the superresolution reconstruction of remote sensing images based on the middle layer supervised convolutional neural network. This paper designs a convolutional neural network with middle layer supervision. There are 16 layers in total, and the seventh layer is designed as an intermediate supervision layer. At present, there are many researches on traditional superresolution reconstruction algorithms and convolutional neural networks, but there are few researches that combine the two together. Convolutional neural network can obtain the high-frequency features of the image and strengthen the detailed information; so, it is necessary to study its application in image reconstruction. This article will separately describe the current research status of image superresolution reconstruction and convolutional neural networks. The middle supervision layer defines the error function of the supervision layer, which is used to optimize the error back propagation mechanism of the convolutional neural network to improve the disappearance of the gradient of the deep convolutional neural network. The algorithm training is mainly divided into four stages: the original remote sensing image preprocessing, the remote sensing image temporal feature extraction stage, the remote sensing image spatial feature extraction stage, and the remote sensing image reconstruction output layer. The last layer of the network draws on the single-frame remote sensing image SRCNN algorithm. The output layer overlaps and adds the remote sensing images of the previous layer, averages the overlapped blocks, eliminates the block effect, and finally obtains high-resolution remote sensing images, which is also equivalent to filter operation. In order to allow users to compare the superresolution effect of remote sensing images more clearly, this paper uses the Qt5 interface library to implement the user interface of the remote sensing image superresolution software platform and uses the intermediate layer convolutional neural network and the remote sensing image superresolution reconstruction algorithm proposed in this paper. When the training epoch reaches 35 times, the network has converged. At this time, the loss function converges to 0.017, and the cumulative time is about 8 hours. This research helps to improve the visual effects of remote sensing images.

Efficient Pyramidal GAN for Versatile Missing Data Reconstruction in Remote Sensing Images

Missing data reconstruction is a classical yet challenging problem in remote sensing image processing due to the complex atmospheric environment and variability of satellite sensors. Most of the contemporary reconstruction methods either handle only one specific task or require supplementary data, while the single-input for multi-task reconstruction has not been explored yet. In this paper we propose a novel Generative Adversarial Network-based unified framework for missing remote sensing image reconstruction, which is capable of various reconstruction tasks given only single source data as input. Specifically, we first propose a Mask Extraction Network (MEN) to obtain a united soft mask, which represents the intrinsic prior under various scenarios and indicates not only location but context information. The versatility of mask extraction enables the multi-task reconstruction of remote sensing images. Besides, we propose a Unified Inpainting Network (UIN) to repair diverse degraded images. Being specifically tailored for remote sensing images, Dilated pyramidal convolutions (DPC) and an Attention Fusion Mechanism (AFM) are introduced to further improve the feature extraction ability and thus exhaustly leveraging the single-input information. Extensive experiments demonstrate the uncompromising performance of the proposed method against state-of-the-art multi-input methods on diverse missing restoration. Moreover, further exploration shows the potential of the proposed method to utilize joint spatio-spectral-temporal information, which is evaluated to outperform existing competitors on remote sense images.

Multi-channel Feature Extraction and Super-Resolution Reconstruction of Remote Sensing Images

Super-resolution reconstruction is an imaging method to improve image resolution. It refers to reconstruct a clear high-resolution image from a low-resolution image. High-resolution remote sensing images can provide more detailed information and higher density, but in the field of remote sensing, because of the limitation of the hardware and vast distances, the remote sensing images are fuzzy sometimes. To facilitate subsequent tasks, this paper proposes a multi-channel feature extraction generative adversarial remote sensing image reconstruction method. According to the characteristics of remote sensing image, a generator is designed, which adds Laplace operator to enhance the edge information of the image, and uses multi-channel feature extraction, which not only enhances the ability of feature extraction but also reduces the number of parameters. In this paper, the super-resolution reconstruction task is carried out based on the 2X magnification factor, and the experimental results are evaluated on SET5/14 and NWPU-RESISC45 dataset. The experimental results show that the images generated by this method have a higher detailed texture and better super-resolution reconstruction effect of remote sensing images.

High-precision Registration Algorithm and Parallel Design Method for High-Resolution Optical Remote Sensing Images

In optical remote sensing image reconstruction, image registration is an important issue to address in order to ensure satisfactory reconstruction performance. In this study, a multi-frame image registration algorithm for high-resolution images and its parallel design method are proposed. The algorithm realizes an improved feature point detection method based on an adaptive gradient bilateral tensor filter and carries out weighted Gaussian surface sub-pixel interpolation to obtain more accurate corner positions, which better guarantees the registration accuracy. On this basis, multi-scale expansion is carried out to generate descriptors for image registration. In addition, the operation-level parallel analysis and design are carried out on a GPU platform based on compute unified device architecture (CUDA), and the memory model of the GPU is utilized reasonably. The task-level parallel analysis and design are carried out based on the GPU stream model. Moreover, based on the open multi-processing (OpenMP) platform, a multi-core CPU carries out parallel design at the operation level and task level, which realizes post-processing operations such as optical remote sensing images loading, accurate matching, and coordinate mapping, thereby effectively improving registration speed. Compared with feature point algorithms and deep learning algorithm, our algorithm and its parallel design significantly improve the registration accuracy and speed of high-resolution optical remote sensing images.

A novel neural network for super-resolution remote sensing image reconstruction

ABSTRACTAn accurate super-resolution image (SR image) reconstruction of remote sensing images (RSI) for preserving quality during the process of super-resolution conversion is crucial for many scientific and operational applications. Recent studies on supervised and unsupervised machine learning methodologies of SR image reconstruction have demonstrated their great potential for higher reconstruction performance in obtaining accuracy and quality. In this paper, a novel neural network with barycentric weight function (BWFNN) is proposed as a non-linear mapping function selected from the features of reference images. The whole process includes an online reconstruction phase and an offline training phase. In these phases, an edge orientation-based pre-learned kernel is introduced to describe and reference prior information, and a simple interpolation-like structure is followed to avoid any conventional iterative computation and lead to fast reconstruction. The innovation of this work is the BWFNN, which uses a non-linear barycentric weight function (BWF) to reconstruct the image details. Compared with most of the conventional reconstruction approaches, the proposed algorithm performs better in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), and the model exhibits significant efficiency in reconstructing the image details.

基于压缩感知的高光谱遥感影像重构方法研究

基于压缩感知的高光谱遥感影像重构方法研究

Convex Optimization Reconstruction of Remote Sensing Images Based on Compressive Sensing and Measurement Matrix

Convex Optimization Reconstruction of Remote Sensing Images Based on Compressive Sensing and Measurement Matrix

Nonlocal Low-Rank-Based Compressed Sensing for Remote Sensing Image Reconstruction

Remote sensing image reconstruction from undersampled data is very much required by the onboard imaging system to cut down data volume and maintain image quality. Nonlocal low-rank regularization deriving from group sparsity, low rank, singular-value thresholding, and nonconvex surrogate functions have recently emerged for image recovery. To use nonlocal low-rank compressed sensing for remote sensing image reconstruction, spectral and temporal redundancy are considered in this letter by utilizing the similarity of correlated bands or historical records. Prior structural knowledge helps to group nonlocal similar blocks more accurately. Oversmoothness of low-rank regularization is improved by injecting referenced structures selectively. The proposed compressed sensing method is tested on satellite images from MODIS, LandSat-7, LandSat-8, IKONOS, and Google Earth to make clear that it outweighs state-of-the-art methods in maintaining fidelity and high visual details.

Reference Information Based Remote Sensing Image Reconstruction with Generalized Nonconvex Low-Rank Approximation

Because of the contradiction between the spatial and temporal resolution of remote sensing images (RSI) and quality loss in the process of acquisition, it is of great significance to reconstruct RSI in remote sensing applications. Recent studies have demonstrated that reference image-based reconstruction methods have great potential for higher reconstruction performance, while lacking accuracy and quality of reconstruction. For this application, a new compressed sensing objective function incorporating a reference image as prior information is developed. We resort to the reference prior information inherent in interior and exterior data simultaneously to build a new generalized nonconvex low-rank approximation framework for RSI reconstruction. Specifically, the innovation of this paper consists of the following three respects: (1) we propose a nonconvex low-rank approximation for reconstructing RSI; (2) we inject reference prior information to overcome over smoothed edges and texture detail losses; (3) on this basis, we combine conjugate gradient algorithms and a single-value threshold (SVT) simultaneously to solve the proposed algorithm. The performance of the algorithm is evaluated both qualitatively and quantitatively. Experimental results demonstrate that the proposed algorithm improves several dBs in terms of peak signal to noise ratio (PSNR) and preserves image details significantly compared to most of the current approaches without reference images as priors. In addition, the generalized nonconvex low-rank approximation of our approach is naturally robust to noise, and therefore, the proposed algorithm can handle low resolution with noisy inputs in a more unified framework.

基于非局部相似性和低秩矩阵的遥感图像重构方法

基于非局部相似性和低秩矩阵的遥感图像重构方法

Compressed sensing based remote sensing image reconstruction via employing similarities of reference images

In the traditional reconstruction algorithm for compressed sensing, we use the measurement matrix and the corresponding observed image to recover the target image. In the application of remote sens...

Sparse-based reconstruction of missing information in remote sensing images from spectral/temporal complementary information

Because of sensor failure and poor observation conditions, remote sensing (RS) images are easily subjected to information loss, which hinders our effective analysis of the earth. As a result, it is of great importance to reconstruct the missing information (MI) of RS images. Recent studies have demonstrated that sparse representation based methods are suitable to fill large-area MI. Therefore, in this paper, we investigate the MI reconstruction of RS images in the framework of sparse representation. Overall, in terms of recovering the MI, this paper makes three major contributions: (1) we propose an analysis model for reconstructing the MI in RS images; (2) we propose to utilize both the spectral and temporal information; and (3) on this basis, we make a detailed comparison of the two kinds of sparse representation models (synthesis model and analysis model). In addition, experiments were conducted to compare the sparse representation methods with the other state-of-the-art methods.