Multi-source Images Research Articles

A hybrid feature fusion deep learning framework for multi-source medical image analysis

Despite the widespread adoption of deep learning to enhance image classification, significant obstacles remain. First, multisource data with diverse sizes and formats is a great challenge for most current deep learning models. Second, lacking manual labeled data for model training limits the application of deep learning. Third, the widely used CNN-based methods shows their limitations in extracting global features and yield poor performance for image topology. To address these issues, we propose a Hybrid Feature Fusion Deep Learning (HFFDL) framework for image classification. This framework consists of an automated image segmentation module, a two-stream backbone module, and a classification module. The automatic image segmentation module utilizes the U-Net model and transfer learning to detect region of interest (ROI) in multisource images; the two-stream backbone module integrates the Swin Transformer architecture with the Inception CNN, with the aim of simultaneous extracting local and global features for efficient representation learning. We evaluate the performance of HFFDL framework with two publicly available image datasets: one for identifying COVID-19 through X-ray scans of the chest (30,386 images), and another for multiclass skin cancer screening using dermoscopy images (25,331 images). The HFFDL framework exhibited greater performance in comparison to many cutting-edge models, achieving the AUC score 0.9835 and 0.8789, respectively. Furthermore, a practical application study conducted in a hospital, identifying viable embryos using medical images, revealed the HFFDL framework outperformed embryologists.

A Fast Sequential Similarity Detection Algorithm for Multi-Source Image Matching

A Fast Sequential Similarity Detection Algorithm for Multi-Source Image Matching

Study on the applicability of FAI linear fitting model in the extraction of cyanobacterial blooms.

Currently, more and more lakes around the world are experiencing outbreaks of cyanobacterial blooms, and high-precision and rapid monitoring of the spatial distribution of algae in water bodies is an important task. Remote sensing technology is one of the effective means for monitoring algae in water bodies. Studies have shown that the Floating Algae Index (FAI) is superior to methods such as the Standardized Differential Vegetation Index (NDVI) and Enhanced Vegetation Index (EVI) in monitoring cyanobacterial blooms. However, compared to the NDVI method, the FAI method has difficulty in determining the threshold, and how to choose the threshold with the highest classification accuracy is challenging. In this study, FAI linear fitting model (FAI-L) is selected to solve the problem that FAI threshold is difficult to determine. Innovatively combine FAI index and NDVI index, and use NDVI index to find the threshold of FAI index. In order to analyze the applicability of FAI-L to extract cyanobacterial blooms, this paper selected multi-temporal Landsat8, HJ-1B, and Sentinel-2 remote sensing images as data sources, and took Chaohu Lake and Taihu Lake in China as research areas to extract cyanobacterial blooms. The results show that (1) the accuracy of extracting cyanobacterial bloom by FAI-L method is generally higher than that by NDVI and FAI. Under different data sources and different research areas, the average accuracy of extracting cyanobacterial blooms by FAI-L method is 95.13%, which is 6.98% and 18.43% higher than that by NDVI and FAI respectively. (2) The average accuracy of FAI-L method for extracting cyanobacterial blooms varies from 84.09 to 99.03%, with a standard deviation of 4.04, which is highly stable and applicable. (3) For simultaneous multi-source image data, the FAI-L method has the highest average accuracy in extracting cyanobacterial blooms, at 95.93%, which is 6.77% and 13.26% higher than NDVI and FAI methods, respectively. In this paper, it is found that FAI-L method shows high accuracy and stability in extracting cyanobacterial blooms, and it can extract the spatial distribution of cyanobacterial blooms well, which can provide a new method for monitoring cyanobacterial blooms.

Multi-Source Image Fusion Based on BEMD and Region Sharpness Guidance Region Overlapping Algorithm

Multi-focal image and multi-modal image fusion technology can fully take advantage of different sensors or different times, retaining the image feature information and improving the image quality. A multi-source image fusion algorithm based on bidimensional empirical mode decomposition (BEMD) and a region sharpness-guided region overlapping algorithm are studied in this article. Firstly, source images are decomposed into multi-layer bidimensional intrinsic mode functions (BIMFs) and residuals from high-frequency layer to low-frequency layer by BEMD. Gaussian bidimensional intrinsic mode functions (GBIMFs) are obtained by applying Gaussian filtering operated on BIMF and calculating the sharpness value of segmented regions using an improved weighted operator based on the Tenengrad function, which is the key to comparison selection and fusion. Then, the GBIMFs and residuals selected by sharpness comparison strategy are fused by the region overlapping method, and the stacked layers are weighted to construct the final fusion image. Finally, based on qualitative evaluation and quantitative evaluation indicators, the proposed algorithm is compared with six typical image fusion algorithms. The comparison results show that the proposed algorithm can effectively capture the feature information of images in different states and reduce the redundant information.

Multi-Source Image Fusion Based Regional Classification Method for Apple Diseases and Pests

Efficient diagnosis of apple diseases and pests is crucial to the healthy development of the apple industry. However, the existing single-source image-based classification methods have limitations due to the constraints of single-source input image information, resulting in low classification accuracy and poor stability. Therefore, a classification method for apple disease and pest areas based on multi-source image fusion is proposed in this paper. Firstly, RGB images and multispectral images are obtained using drones to construct an apple diseases and pests canopy multi-source image dataset. Secondly, a vegetation index selection method based on saliency attention is proposed, which uses a multi-label ReliefF feature selection algorithm to obtain the importance scores of vegetation indices, enabling the automatic selection of vegetation indices. Finally, an apple disease and pest area multi-label classification model named AMMFNet is constructed, which effectively combines the advantages of RGB and multispectral multi-source images, performs data-level fusion of multi-source image data, and combines channel attention mechanisms to exploit the complementary aspects between multi-source data. The experimental results demonstrated that the proposed AMMFNet achieves a significant subset accuracy of 92.92%, a sample accuracy of 85.43%, and an F1 value of 86.21% on the apple disease and pest multi-source image dataset, representing improvements of 8.93% and 10.9% compared to prediction methods using only RGB or multispectral images. The experimental results also proved that the proposed method can provide technical support for the coarse-grained positioning of diseases and pests in apple orchards and has good application potential in the apple planting industry.

Multi-Source Image Matching Algorithms for UAV Positioning: Benchmarking, Innovation, and Combined Strategies

The accuracy and reliability of unmanned aerial vehicle (UAV) visual positioning systems are dependent on the performance of multi-source image matching algorithms. Despite many advancements, targeted performance evaluation frameworks and datasets for UAV positioning are still lacking. Moreover, existing consistency verification methods such as Random Sample Consensus (RANSAC) often fail to entirely eliminate mismatches, affecting the precision and stability of the matching process. The contributions of this research include the following: (1) the development of a benchmarking framework accompanied by a large evaluation dataset for assessing the efficacy of multi-source image matching algorithms; (2) the results of this benchmarking framework indicate that combinations of multiple algorithms significantly enhance the Match Success Rate (MSR); (3) the introduction of a novel Geographic Geometric Consistency (GGC) method that effectively identifies mismatches within RANSAC results and accommodates rotational and scale variations; and (4) the implementation of a distance threshold iteration (DTI) method that, according to experimental results, achieves an 87.29% MSR with a Root Mean Square Error (RMSE) of 1.11 m (2.22 pixels) while maintaining runtime at only 1.52 times that of a single execution, thus optimizing the trade-off between MSR, accuracy, and efficiency. Furthermore, when compared with existing studies on UAV positioning, the multi-source image matching algorithms demonstrated a sub-meter positioning error, significantly outperforming the comparative method. These advancements are poised to enhance the application of advanced multi-source image matching technologies in UAV visual positioning.

Walking representation and simulation based on multi-source image fusion and multi-agent reinforcement learning for gait rehabilitation

In the formulation of strategies for walking rehabilitation, achieving precise identification of the current state and making rational predictions about the future state are crucial but often unrealized. To tackle this challenge, our study introduces a unified framework that integrates a novel 3D walking motion capture method using multi-source image fusion and a walking rehabilitation simulation approach based on multi-agent reinforcement learning. We found that, (i) the proposal achieved an accurate 3D walking motion capture and outperforms other advanced methods. Experimental evidence indicates that, compared to similar visual skeleton tracking methods, the proposed approach yields results with higher Pearson correlation (r=0.93), intra-class correlation coefficient (ICC(2,1)=0.91), and narrower confidence intervals ([0.90,0.95] for r, [0.88,0.94] for ICC(2,1)) when compared to standard results. The outcomes of the proposed approach also exhibit commendable correlation and concurrence with those obtained through the IMU-based skeleton tracking method in the assessment of gait parameters ([0.85,0.89] for r, [0.75,0.81] for ICC(2,1)); (ii) multi-agent reinforcement learning has the potential to be used to solve the simulation task of gait rehabilitation. In mimicry experiment, our proposed simulation method for gait rehabilitation not only enables the intelligent agent to converge from the initial state to the target state, but also observes evolutionary patterns similar to those observed in clinical practice through motor state resolution. This study offers valuable contributions to walking rehabilitation, enabling precise assessment and simulation-based interventions, with potential implications for clinical practice and patient outcomes.

MATI: Multimodal Adaptive Tracking Integrator for Robust Visual Object Tracking.

Visual object tracking, pivotal for applications like earth observation and environmental monitoring, encounters challenges under adverse conditions such as low light and complex backgrounds. Traditional tracking technologies often falter, especially when tracking dynamic objects like aircraft amidst rapid movements and environmental disturbances. This study introduces an innovative adaptive multimodal image object-tracking model that harnesses the capabilities of multispectral image sensors, combining infrared and visible light imagery to significantly enhance tracking accuracy and robustness. By employing the advanced vision transformer architecture and integrating token spatial filtering (TSF) and crossmodal compensation (CMC), our model dynamically adjusts to diverse tracking scenarios. Comprehensive experiments conducted on a private dataset and various public datasets demonstrate the model's superior performance under extreme conditions, affirming its adaptability to rapid environmental changes and sensor limitations. This research not only advances visual tracking technology but also offers extensive insights into multisource image fusion and adaptive tracking strategies, establishing a robust foundation for future enhancements in sensor-based tracking systems.

MFPANet: Multi-Scale Feature Perception and Aggregation Network for High-Resolution Snow Depth Estimation

Accurate snow depth estimation is of significant importance, particularly for preventing avalanche disasters and predicting flood seasons. The predominant approaches for such snow depth estimation, based on deep learning methods, typically rely on passive microwave remote sensing data. However, due to the low resolution of passive microwave remote sensing data, it often results in low-accuracy outcomes, posing considerable limitations in application. To further improve the accuracy of snow depth estimation, in this paper, we used active microwave remote sensing data. We fused multi-spectral optical satellite images, synthetic aperture radar (SAR) images and land cover distribution images to generate a snow remote sensing dataset (SRSD). It is a first-of-its-kind dataset that includes active microwave remote sensing images in high-latitude regions of Asia. Using these novel data, we proposed a multi-scale feature perception and aggregation neural network (MFPANet) that focuses on improving feature extraction from multi-source images. Our systematic analysis reveals that the proposed approach is not only robust but also achieves high accuracy in snow depth estimation compared to existing state-of-the-art methods, with RMSE of 0.360 and with MAE of 0.128. Finally, we selected several representative areas in our study region and applied our method to map snow depth distribution, demonstrating its broad application prospects.

Mapping global mangrove canopy height by integrating Ice, Cloud, and Land Elevation Satellite-2 photon-counting LiDAR data with multi-source images

Large-scale and precise measurement of mangrove canopy height is crucial for understanding and evaluating wetland ecosystems' condition, health, and productivity. This study generates a global mangrove canopy height map with a 30 m resolution by integrating Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) photon-counting light detection and ranging (LiDAR) data with multi-source imagery. Initially, high-quality mangrove canopy height samples were extracted using meticulous processing and filtering of ICESat-2 data. Subsequently, mangrove canopy height models were established using the random forest (RF) algorithm, incorporating ICESat-2 canopy height samples, Sentinel-2 data, TanDEM-X DEM data and WorldClim data. Furthermore, a global 30 m mangrove canopy height map was generated utilizing the Google Earth Engine platform. Finally, the global map's accuracy was evaluated by comparing it with reference canopy heights derived from both space-borne and airborne LiDAR data. Results indicate that the global 30 m resolution mangrove height map was found to be consistent with canopy heights obtained from space-borne (r = 0.88, Bisa = −0.07 m, RMSE = 3.66 m, RMSE% = 29.86 %) and airborne LiDAR (r = 0.52, Bisa = −1.08 m, RMSE = 3.39 m, RMSE% = 39.05 %). Additionally, our findings reveal that mangroves worldwide exhibit an average height of 12.65 m, with the tallest mangrove reaching a height of 44.94 m. These results demonstrate the feasibility and effectiveness of using ICESat-2 data integrated with multi-source imagery to generate a global mangrove canopy height map. This dataset offers reliable information that can significantly support government and organizational efforts to protect and conserve mangrove ecosystems.

Parallel Hierarchical Multi-View Feature Fusion Based on Canonical Correlation Analysis for Mammogram Retrieval

Due to the diversity of image sources, content-based multi-source image fusion and retrieval have shown promising capabilities in computer vision tasks, and especially when applied in Computer-Aided Diagnosis (CAD) to automate and improve the accuracy of medical image analysis. The combination of computer vision and CAD systems has the potential to revolutionize healthcare by augmenting the expertise of clinicians, improving overall diagnostic accuracy and helping experts in the clinical decision-making process by classifying and retrieving similar annotated clinical images to a given query. In the context of multi-view mammography interpretation, the concept of multi-view feature fusion has recently been studied to improve retrieval performance while effectively guaranteeing the complementarity of both MLO and CC views. However, conventional multi-view feature fusion makes descriptors long and lacks to take into consideration the relationship between descriptors. To deal with this issue, we propose two hierarchical multi-view feature fusion methods, for multi-view mammogram retrieval, based on the Canonical Correlation Analysis (CCA), which is the most commonly used multivariate parametric test. In fact, we have adapted CCA to determine the relationship between two descriptors by processing latent correlation factors. Moreover, after extracting descriptors for each view, a comparative study of texture and shape fusion descriptors is proposed in order to identify the more discriminative features for multi-view mammogram retrieval. Then, a query-dependent distance metric preserving both visual resemblance and semantic similarity is carried out to dynamically determine the more appropriate distance measure for each query image. Extensive experiments on the challenging Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) have demonstrated the effectiveness of the proposed hierarchical multi-view feature fusion for mammogram retrieval, which outperforms the performance achieved either by conventional fused information or by single view information. To improve the transparency of our paper, the source code of the proposed method and the related dataset (including readme files) are publicly accessible through the following GitHub link: https://github.com/ABDERRAHIMMAR/Multi-View-Feature-Fusion-for-Mammogram-Retrieval . This open-access resource empowers researchers and practitioners to delve deeper into our methodology, fostering collaboration and advancements in the field of computer-aided diagnosis and medical image analysis.

MuST: Robust Image Watermarking for Multi-Source Tracing

In recent years, with the popularity of social media applications, massive digital images are available online, which brings great convenience to image recreation. However, the use of unauthorized image materials in multi-source composite images is still inadequately regulated, which may cause significant loss and discouragement to the copyright owners of the source image materials. Ideally, deep watermarking techniques could provide a solution for protecting these copyrights based on their encoder-noise-decoder training strategy. Yet existing image watermarking schemes, which are mostly designed for single images, cannot well address the copyright protection requirements in this scenario, since the multi-source image composing process commonly includes distortions that are not well investigated in previous methods, e.g., the extreme downsizing. To meet such demands, we propose MuST, a multi-source tracing robust watermarking scheme, whose architecture includes a multi-source image detector and minimum external rectangle operation for multiple watermark resynchronization and extraction. Furthermore, we constructed an image material dataset covering common image categories and designed the simulation model of the multi-source image composing process as the noise layer. Experiments demonstrate the excellent performance of MuST in tracing sources of image materials from the composite images compared with SOTA watermarking methods, which could maintain the extraction accuracy above 98% to trace the sources of at least 3 different image materials while keeping the average PSNR of watermarked image materials higher than 42.51 dB. We released our code on https://github.com/MrCrims/MuST

Automatic Pavement Crack Detection in Multisource Fusion Images Using Similarity and Difference Features

In urban pavement crack detection, the single detection platform with visible light sensors is mostly used for detection. Visible light sensors are highly susceptible to lighting conditions. In low-light environments or areas with shadows over the cracks, this may result in false positives or missed detections of pavement cracks. The paper utilizes an image acquisition system that integrates multiple detection platforms, including visible light and infrared sensors, to enhance both the efficiency and accuracy of pavement crack detection. We propose an automatic pavement crack detection approach that uses the similarities and differences between visible light images and infrared image features (MSFSD) to achieve all-weather, real-time detection of pavement cracks. MSFSD mainly includes multi-source image fusion and pixel-level crack detection. In multi-source image fusion, a generative adversarial network (C-GAN) with skip connections and dilated convolutional blocks (CRes2Net blocks) is constructed. The structural similarity index (SSIM) function and the sum of the correlations of differences (SCD) function are used as the loss function of the generator, allowing the fusion image to retain more details and edge information in the source image. In pixel-level crack detection, a crack detection model is constructed, and CRes2Net is used as the core network to obtain pixel-level labels and perform semantic segmentation. It is verified by experiments that MSFSD achieves 85.1% precision and 88.4% recall in different types of crack detection. The MIoU and detection efficiency of MSFSD are 9% and 48% higher than other methods, respectively.

High spatiotemporal-resolution mapping for a seasonal erosion flooding inundation using time-series Landsat and MODIS images

Seasonal erosion flooding events present a significant challenge for effective disaster monitoring and land degradation studies. This research addresses this challenge by harnessing the combined capabilities of time-series Landsat and MODIS images to achieve high spatiotemporal-resolution mapping of flooding during such events. The study underscores the critical importance of precise flood monitoring for disaster mitigation and informed land management. To overcome the limitations posed by the trade-off between spatial and temporal resolution in current satellite sensors, we emplyedand theflexible spatiotemporal data fusion (FSDAF) methods to produce synthetic flood images with enhanced spatiotemporal resolutions for mapping by using MODIS and Landsat data from August 29 to September 3, 2016. A comparison was made between flood maps from several post-disaster forecasts based on ground-obtained time-series images of the Tumen River flood in China. According to the FSDAF approach, the input Landsat image of March 25, 2016, and the fused results had a root mean square error (RMSE) of 0.0301, average difference of 0.001, r of 0.941, and structure similarity indexof 0.939, indicating that temporal variation data had been effectively incorporated into a forecast on August 16, 2016. Results also indicated that the FSDAF forecast values are lower than those from the actual Landsat image. The results of the study also showed that the generated images could be effectively used for flood mapping. By using our newly developed simulation model, we were able to produce a comprehensive map of the inundated areas during the event from August 29 to September 3, 2016. This shows that FSDAF holds great potential for flood prediction and study and has the potential to benefit further disaster-related land degradation by combining multi-source images to provide high temporal and spatial resolution remote sensing information.

Probabilistic Estimation of Tropical Cyclone Intensity Based on Multi-Source Satellite Remote Sensing Images

Estimating the intensity of tropical cyclones (TCs) is beneficial for preventing and reducing the impact of natural disasters. Most existing methods for estimating TC intensity utilize single-satellite or single-band remote sensing images, but they lack the ability to quantify the uncertainty of the estimation results. However, TC, as a typical chaotic system, often requires confidence intervals for intensity estimates in real-world emergency decision-making scenarios. Additionally, the use of multi-source image inputs contributes to the uncertainty of the model. Consequently, this study introduces a neural network (MTCIE) that utilizes multi-source satellite images to provide probabilistic estimates of TC intensity. The model utilizes infrared and microwave images from multiple satellites as inputs. It uses a dual-branch self-attention encoder to extract TC image features and provides uncertainty estimates for TC intensity. Furthermore, a dataset for estimating the intensity of multi-source TC remote sensing images (MTCID) is constructed through the registration of latitude, longitude, and time, along with data augmentation. The proposed method achieves a MAE of 7.42 kt in deterministic estimation, comparable to mainstream networks like TCIENet. In uncertain estimation, it outperforms methods like MC Dropout in the PICP metric, providing reliable probability estimates. This supports TC disaster emergency decision making, enhancing risk mitigation in real-world applications.

SharDif: Sharing and Differential Learning for Image Fusion.

Image fusion is the generation of an informative image that contains complementary information from the original sensor images, such as texture details and attentional targets. Existing methods have designed a variety of feature extraction algorithms and fusion strategies to achieve image fusion. However, these methods ignore the extraction of common features in the original multi-source images. The point of view proposed in this paper is that image fusion is to retain, as much as possible, the useful shared features and complementary differential features of the original multi-source images. Shared and differential learning methods for infrared and visible light image fusion are proposed. An encoder with shared weights is used to extract shared common features contained in infrared and visible light images, and the other two encoder blocks are used to extract differential features of infrared images and visible light images, respectively. Effective learning of shared and differential features is achieved through weight sharing and loss functions. Then, the fusion of shared features and differential features is achieved via a weighted fusion strategy based on an entropy-weighted attention mechanism. The experimental results demonstrate the effectiveness of the proposed model with its algorithm. Compared with the-state-of-the-art methods, the significant advantage of the proposed method is that it retains the structural information of the original image and has better fusion accuracy and visual perception effect.

Relationship Learning from Multisource Images via Spatial-spectral Perception Network.

Advances in multisource remote sensing have allowed for the development of more comprehensive observation. The adoption of deep convolutional neural networks (CNN) naturally includes spatial-spectral information, which has achieved promising performance in multisource data classification. However, challenges are still found with the extraction of spatial distribution and spectrum relationships, which eventually limit the classification performance. To solve the issue, a spatial-spectral perception network (S2PNet) is proposed to extract the advantages of different data sources and the cross information between data sources in a targeted manner. Specifically, the spatial perception network is developed to build the spatial distribution relationship from high-resolution images, while the spectral perception network extracts the spectrum relationship from spectral images. For perceiving cross information, a memory unit is utilized to store the features from different data sources in succession. In addition, the distance loss and reconstruction loss are introduced to keep the feature integrity, and the cross-entropy loss ensures that features can distinguish different classes. The comprehensive experiments are conducted on several datasets to validate the superiority of the proposed algorithm. The proposed S2PNet outperforms the considered classifiers with an average improvement of +0.77%, +5.62%, +1.58%, and +1.79% for overall accuracy values.

Representation-Enhanced Status Replay Network for Multisource Remote-Sensing Image Classification.

Deep-learning-based methods are widely used in multisource remote-sensing image classification, and the improvement in their performance confirms the effectiveness of deep learning for classification tasks. However, the inherent underlying problems of deep-learning models still hinder the further improvement of classification accuracy. For example, after multiple rounds of optimization learning, representation bias and classifier bias are accumulated, which prevents the further optimization of network performance. In addition, the imbalance of fusion information among multisource images also leads to insufficient information interaction throughout the fusion process, thus making it difficult to fully utilize the complementary information of multisource data. To address these issues, a Representation-enhanced Status Replay Network (RSRNet) is proposed. First, a dual augmentation including modal augmentation and semantic augmentation is proposed to enhance the transferability and discreteness of feature representation, to reduce the impact of representation bias in the feature extractor. Then, to alleviate the classifier bias and maintain the stability of the decision boundary, a status replay strategy (SRS) is built to regulate the learning and optimization of the classifier. Finally, aiming to improve the interactivity of modal fusion, a novel cross-modal interactive fusion (CMIF) method is employed to jointly optimize the parameters of different branches by combining multisource information. Quantitative and qualitative results on three datasets demonstrate the superiority of RSRNet in multisource remote-sensing image classification, and its outperformance compared with other state-of-the-art methods.

A Principle Design of Registration-Fusion Consistency: Toward Interpretable Deep Unregistered Hyperspectral Image Fusion.

For hyperspectral image (HSI) and multispectral image (MSI) fusion, it is often overlooked that multisource images acquired under different imaging conditions are difficult to be perfectly registered. Although some works attempt to fuse unregistered images, two thorny challenges remain. One is that registration and fusion are usually modeled as two independent tasks, and there is no yet a unified physical model to tightly couple them. Another is that deep learning (DL)-based methods may lack sufficient interpretability and generalization. In response to the above challenges, we propose an unregistered HSI fusion framework energized by a unified model of registration and fusion. First, a novel registration-fusion consistency physical perception model (RFCM) is designed, which uniformly models the image registration and fusion problem to greatly reduce the sensitivity of fusion performance to registration accuracy. Then, an HSI fusion framework (MoE-PNP) is proposed to learn the knowledge reasoning process for solving RFCM. Each basic module of MoE-PNP one-to-one corresponds to the operation in the optimization algorithm of RFCM, which can ensure clear interpretability of the network. Moreover, MoE-PNP captures the general fusion principle for different unregistered images and therefore has good generalization. Extensive experiments demonstrate that MoE-PNP achieves state-of-the-art performance for unregistered HSI and MSI fusion. The code is available at https://github.com/Jiahuiqu/MoE-PNP.

VL-MFL: UAV Visual Localization Based on Multisource Image Feature Learning

VL-MFL: UAV Visual Localization Based on Multisource Image Feature Learning