Multi-source Remote Sensing Image Fusion Research Articles

Multisource remote sensing image fusion processing in plateau seismic region feature information extraction and application analysis – An example of the Menyuan Ms6.9 earthquake on January 8, 2022

Abstract A 6.9 magnitude earthquake hit Menyuan County, Haibei Prefecture, Qinghai Province, at 01:45 PM Beijing time on January 8, 2022 (17:45 PM GMT time on January 7, 2022). To explore the magnitude of the earthquake deformation and the affected area, this work combined optical remote sensing interpretation, interferometric synthetic aperture radar (InSAR) coseismic deformation extraction, and field surveys for research and analysis. Relying on the high-resolution Earth observation system of the Qinghai Remote Sensing Center for Natural Resources, high-resolution GF1D, GF2, and TRIPLESAT optical remote sensing images were acquired immediately after the earthquake. The airborne triangulation encryption method was used to carry out orthographic correction, fusion, and mosaic processing of digital orthophoto map (DOM) and digital surface model (DSM) images, and first-hand optical remote sensing images of the disaster areas were obtained. Based on differential InSAR (D-InSAR), small baseline subset InSAR (SBAS-InSAR) and lifting rail fusion methods, the coseismic deformation field and deformation rate of the lifting rail direction were obtained by using Sentinel-1A data processing before and after the earthquake. Combined with optical interpretation, InSAR deformation, and field investigation, the results show that the deformation trend of the line of sight (LOS) images to the north and south of the ascending and descending orbits show an obvious opposite trend. The surface shape variables are −50 to 45 cm and −65 to 72 cm, respectively, and the deformation rate before the earthquake reached 25 cm/year. The deformation field characteristics show that the earthquake was mainly due to thrust, and the coseismic deformation field fractured along the WNW‒ESE direction with a length of approximately 33 km. The areas affected by 10 mm, 20 cm, and 50 cm deformation magnitudes in the whole earthquake area were 975.14, 321.10, and 38.55 km2, respectively. Within 20 km, there were two main affected townships, namely, Sujitan Township and Huangcheng Mongolian Township. Within 50 km, there were four main affected towns and townships, namely, Sujitan Township, Mongolian Township of the Imperial city, Qingshizui town, and Haomen town.

A Joint Convolutional Cross ViT Network for Hyperspectral and Light Detection and Ranging Fusion Classification

The fusion of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data for classification has received widespread attention and has led to significant progress in research and remote sensing applications. However, existing common CNN architectures suffer from the significant drawback of not being able to model remote sensing images globally, while transformer architectures are not able to capture local features effectively. To address these bottlenecks, this paper proposes a classification framework for multisource remote sensing image fusion. First, a spatial and spectral feature projection network is constructed based on parallel feature extraction by combining HSI and LiDAR data, which is conducive to extracting joint spatial, spectral, and elevation features from different source data. Furthermore, in order to construct local–global nonlinear feature mapping more flexibly, a network architecture coupling together multiscale convolution and a multiscale vision transformer is proposed. Moreover, a plug-and-play nonlocal feature token aggregation module is designed to adaptively adjust the domain offsets between different features, while a class token is employed to reduce the complexity of high-dimensional feature fusion. On three open-source remote sensing datasets, the performance of the proposed multisource fusion classification framework improves about 1% to 3% over other state-of-the-art algorithms.

Enhancing Road Extraction in Large-Scale Complex Terrain through Multi-Source Remote Sensing Image Fusion and Optimization

In large-scale remote sensing scenarios characterized by intricate terrain, the straightforward road imaging features in synthetic aperture radar (SAR) images make them susceptible to interference from other elements such as ridges, compromising the robustness of conventional SAR image road extraction methods. This paper introduces a method that integrates Gaofen-3 (GF-3) with a resolution of 3.0 m, Digital Elevation Models (DEMs), and Gaofen-2 (GF-2) remote sensing image data with a resolution of 4.0 m, aiming to improve the performance of road extraction in complex terrain. Leveraging DEMs, this study addresses the limitations in feature-based SAR algorithms, extending their application to complex remote sensing scenarios. Decision-level fusion, integrating SAR and multispectral images, further refines road extraction precision. To overcome issues related to terrain interference, including fragmented road segments, an adaptive rotated median filter and graph-theory-based optimization are introduced. These advancements collectively enhance road recognition accuracy and topological precision. The experimental results validate the effectiveness of the multi-source remote sensing image fusion and optimization methods. Compared to road extraction from multispectral images, the F1-score of the proposed method on the test images increased by 2.18%, 4.22%, and 1.4%, respectively.

Improved GSA Method Based on Deep Convolutional Features

With the rapid development of remote sensing platforms, sensors and other technologies, remote sensing, as an important technical means of collecting information on land cover and its changes, plays an important role in land cover classification and dynamic monitoring. Due to the limitation of the imaging mechanism of in-orbit optical remote sensing sensors, it is difficult for to take into account both spatial resolution and spectral resolution. There is a need for complementary use of different sensor images and to improve the accuracy of feature interpretation. In this study, based on the Gram-Schmidt Adaptive (GSA) and its classical extension, a neural network-based multi-source remote sensing image fusion method is proposed by combining the U-Net coding and decoding structure with the Non-Local spatial attention mechanism. Using two sets of experimental datasets, three kinds of fusion evaluation metrics without reference and three kinds of fusion evaluation metrics with reference, the improved method and the existing fusion method are compared.

Multiresolution generative adversarial networks with bidirectional adaptive-stage progressive guided fusion for remote sensing image

ABSTRACT Remote sensing image (RSI) with concurrently high spatial, temporal, and spectral resolutions cannot be produced by a single sensor. Multisource RSI fusion is a convenient technique to realize high spatial resolution multispectral (MS) images (spatial spectral fusion, i.e. SSF) and high temporal and spatial resolution MS images (spatiotemporal fusion, i.e. STF). Currently, deep learning-based fusion models can only implement SSF or STF, lacking models that perform both SSF and STF. Multiresolution generative adversarial networks with bidirectional adaptive-stage progressive guided fusion (BAPGF) for RSI are proposed to implement both SSF and STF, namely BPF-MGAN. A bidirectional adaptive-stage feature extraction architecture in fine-scale-to-coarse-scale and coarse-scale-to-fine-scale modes is introduced. The designed BAPGF introduces a previous fusion result-oriented cross-stage-level dual-residual attention fusion strategy to enhance critical information and suppress superfluous information. Adaptive resolution U-shaped discriminators are implemented to feed multiresolution context into the generator. A generalized multitask loss function unlimited by no-reference images is developed to strengthen the model via constraints on the multiscale feature, structural, and content similarities. The BPF-MGAN model is validated on SSF datasets and STF datasets. Compared with the state-of-the-art SSF and STF models, results demonstrate the superior performance of the proposed BPF-MGAN model in both subjective and objective evaluations.

Study on characteristic maps of multi-spectral remote sensing images of geological hazards

Geological disasters, as the main problems facing urban construction and development, directly threaten the life and property of social residents and living environment, with sudden, wide distribution, destruction and other characteristics. Nowadays, the traditional ground monitoring technology has been unable to meet the needs of geological disaster monitoring research in the new era. Therefore, on the basis of mastering the traditional monitoring experience, scholars from various countries put forward the use of multi-spectral remote sensing image data to obtain the data information of geological disasters in various regions, study the main causes and distribution rules of geological disasters, and put forward effective prediction models and solutions as soon as possible. On the basis of understanding the current situation of geological disaster monitoring and prevention work in China, this paper uses multi-source remote sensing image fusion technology to deeply study the multi-spectral remote sensing image feature map of geological disaster and clarify the relationship between different types of remote sensing image features, so as to provide effective basis for earthquake disaster monitoring and prevention work.

Unsupervised Pansharpening Method Using Residual Network With Spatial Texture Attention

Recently, deep learning has become one of the most popular tools for pansharpening, many relevant methods have been investigated and reflected great performance. However, a non-negligible problem is the absence of ground-truth (GT). A common solution is using degraded images as training input and the original images are employed as GT. The learned mapping between low resolution (LR) and high resolution (HR) is simulated, is not real, which may cause spectral distortion or insufficient spatial texture enhancement of fused images. In order to address the drawback, a novel unsupervised attention pansharpening net (UAP-Net) is proposed. The proposed UAP-Net mainly contains two major components: 1) the deep residual network (DRN) and 2) spatial texture attention block (STAB). The DRN aims to extract spectral features and spatial details features from low-resolution multi-spectral (LRMS) and panchromatic (PAN), and to fuse those features to make them more representative. The designed STAB adopts the high-frequency component of corresponding input PAN as the weight to enhance the spatial details of the residual block output features. Moreover, a new loss function including two spatial losses and two spectral losses are established. The losses are calculated in the spatial domain and the frequency domain, respectively. Experiments on Gaofen-2 and Worldview-2 remote sensing data demonstrate that the proposed UAP-Net could fuse PAN and LRMS images effectively without the help of high-resolution multi-spectral (HRMS). The proposed framework is fully general and can be used for many multisource remote sensing image fusion, and achieves optimal performance in terms of both the subjective visual effect and the quantitative evaluation.

Remote Monitoring of NH3-N Content in Small-Sized Inland Waterbody Based on Low and Medium Resolution Multi-Source Remote Sensing Image Fusion

In applying quantitative remote sensing in water quality monitoring for small inland rivers, the time-frequency of monitoring dramatically impacts the accuracy of time-spatial changes estimates of the water quality parameters. Due to the limitation of satellite sensor design and the influence of atmospheric conditions, the number of spatiotemporal dynamic monitoring images of water quality parameters is insufficient. Meanwhile, MODIS and other high temporal resolution images’ spatial resolution is too low to effectively extract small inland river boundaries. To solve the problem, many researchers used Spatio-temporal fusion models in multisource data remote sensing monitoring of ground features. The wildly used Spatio-temporal fusion models, such as FSDAF (flexible spatial-temporal data fusion), have poor performance in heterogeneous changes of ground objects. We proposed a spatiotemporal fusion algorithm SR-FSDAF (Super-resolution based flexible spatiotemporal data fusion) to solve the problem. Based on the FSDAF, it added ESPCN to reconstruct the spatial change prediction image, so as to obtain better prediction results for heterogeneous changes. Both qualitative and quantitative evaluation results showed that our fusion algorithm obtained better results. We compared the band sensitivity of the images before and after fusion to find out that the sensitive band combination of NH3-N has not changed, which proved that the fusion method can be used to improve the time-frequency of NH3-N inversion. After the fusion, we compared the accuracy of linear regression and random forest inversion models and selected the random forest model with better accuracy to predict the NH3-N concentration. The inversion accuracy of NH3-N was as follows: the R2 was 0.75, the MAPE was 23.7% and the RMSE was 0.15. The overall concentration change trend of NH3-N in the study area was high-water period < water-stable period < low water period. NH3-N pollution was serious in some reaches.

Multi-Source Remote Sensing Image Fusion for Ship Target Detection and Recognition

The active recognition of interesting targets has been a vital issue for remote sensing. In this paper, a novel multi-source fusion method for ship target detection and recognition is proposed. By introducing synthetic aperture radar (SAR) sensor images, the proposed method solves the problem of precision degradation in optical remote sensing image target detection and recognition caused by the limit of illumination and weather conditions. The proposed method obtains port slice images containing ship targets by fusing optical data with SAR data. On this basis, spectral residual saliency and region growth method are used to detect ship targets in optical image, while SAR data are introduced to improve the accuracy of ship detection based on joint shape analysis and multi-feature classification. Finally, feature point matching, contour extraction and brightness saliency are used to detect the ship parts, and the ship target types are identified according to the voting results of part information. The proposed ship detection method obtained 91.43% recognition accuracy. The results showed that this paper provides an effective and efficient ship target detection and recognition method based on multi-source remote sensing images fusion.

Assessment of Multisource Remote Sensing Image Fusion by several dissimilarity Methods

Recently, advancements in remote sensing technology have made it easier to obtain various temporal and spatial resolution satellite data. Remote sensing techniques can be a useful tool to detect vegetation and soil conditions, monitor crop diseases and natural disaster prevention, etc. Although the same scene taken by different sensors belong to the same ground object, the information that they offered are redundant, complementary and collaborative due to the spatial, spectral and temporal resolution are different. The method of image fusion can integrate an image with rich details and valuable information from multi-source remote sensing images, which aim to obtain more comprehensive and precise observations of the ground object. By using aspects from multi-source image fusion, this review presents the current status and future trends in remote sensing image fusion. First, different image properties and their applications are presented for remote sensing datasets at home and abroad. Second, a general summary and inductive analysis of the challenging difficulty of different types of multisource image fusion methods is conducted. Third, experiments are tested on eight different methodological approaches, and experimental results demonstrate that GSA method is the best alternative in terms of obtaining high spatial resolution and retaining the spectral information.

Saliency-Guided Nonsubsampled Shearlet Transform for Multisource Remote Sensing Image Fusion.

The rapid development of remote sensing and space technology provides multisource remote sensing image data for earth observation in the same area. Information provided by these images, however, is often complementary and cooperative, and multisource image fusion is still challenging. This paper proposes a novel multisource remote sensing image fusion algorithm. It integrates the contrast saliency map (CSM) and the sum-modified-Laplacian (SML) in the nonsubsampled shearlet transform (NSST) domain. The NSST is utilized to decompose the source images into low-frequency sub-bands and high-frequency sub-bands. Low-frequency sub-bands reflect the contrast and brightness of the source images, while high-frequency sub-bands reflect the texture and details of the source images. Using this information, the contrast saliency map and SML fusion rules are introduced into the corresponding sub-bands. Finally, the inverse NSST reconstructs the fusion image. Experimental results demonstrate that the proposed multisource remote image fusion technique performs well in terms of contrast enhancement and detail preservation.

Development status and future prospects of multi-source remote sensing image fusion

Development status and future prospects of multi-source remote sensing image fusion

Dynamic Monitoring of Forest Land in Fuling District Based on Multi-Source Time Series Remote Sensing Images

Time series remote sensing images can be used to monitor the dynamic changes of forest lands. Due to consistent cloud cover and fog, a single sensor typically provides limited data for dynamic monitoring. This problem is solved by combining observations from multiple sensors to form a time series (a satellite image time series). In this paper, the pixel-based multi-source remote sensing image fusion (MulTiFuse) method is applied to combine the Landsat time series and Huanjing-1 A/B (HJ-1 A/B) data in the Fuling district of Chongqing, China. The fusion results are further corrected and improved with spatial features. Dynamic monitoring and analysis of the study area are subsequently performed on the improved time series data using the combination of Mann-Kendall trend detection method and Theil Sen Slope analysis. The monitoring results show that a majority of the forest land (60.08%) has experienced strong growth during the 1999–2013 period. Accuracy assessment indicates that the dynamic monitoring using the fused image time series produces results with relatively high accuracies.

A variational method for multisource remote-sensing image fusion

With the increasing availability of multisource image data from Earth observation satellites, image fusion, a technique that produces a single image which preserves major salient features from a set of different inputs, has become an important tool in the field of remote sensing since usually the complete information cannot be obtained by a single sensor. In this article, we develop a new pixel-based variational model for image fusion using gradient features. The basic assumption is that the fused image should have a gradient that is close to the most salient gradient in the multisource inputs. Meanwhile, we integrate the inputs with the average quadratic local dispersion measure for the purpose of uniform and natural perception. Furthermore, we introduce a split Bregman algorithm to implement the proposed functional more effectively. To verify the effect of the proposed method, we visually and quantitatively compare it with the conventional image fusion schemes, such as the Laplacian pyramid, morphological pyramid, and geometry-based enhancement fusion methods. The results demonstrate the effectiveness and stability of the proposed method in terms of the related fusion evaluation benchmarks. In particular, the computation efficiency of the proposed method compared with other variational methods also shows that our method is remarkable.

Multi-Source Remote Sensing Image Fusion Algorithm Based on Combined Wavelet Transform and HIS Transform

Multi-Source Remote Sensing Image Fusion Algorithm Based on Combined Wavelet Transform and HIS Transform

Multi-source remote sensing image fusion based on improved wavelet transform method

SPOT panchromatic images and multi-spectrum TM remote sensing images were fused by an improved wavelet transform adopting the advantages of direction adjusted wavelet transform and inseparable frame wavelet transform with the aid of Matlab and ERDAS IMAGINE.Compared with traditional fusion methods,this method can maintain both the spectrum information of the multi-spectrum images and the spatial resolution characteristic of the full-color images,therefore enhance the ability of phantom interpretation.

Multi-source remote sensing image fusion based on support vector machine

Remote Sensing image fusion is an effective way to use the large volume of data from multi-source images. This paper introduces a new method of remote sensing image fusion based on support vector machine (SVM), using high spatial resolution data SPIN-2 and multi-spectral remote sensing data SPOT-4. Firstly, the new method is established by building a model of remote sensing image fusion based on SVM. Then by using SPIN-2 data and SPOT-4 data, image classification fusion is tested. Finally, an evaluation of the fusion result is made in two ways. 1) From subjectivity assessment, the spatial resolution of the fused image is improved compared to the SPOT-4. And it is clearly that the texture of the fused image is distinctive. 2) From quantitative analysis, the effect of classification fusion is better. As a whole, the result shows that the accuracy of image fusion based on SVM is high and the SVM algorithm can be recommended for application in remote sensing image fusion processes.