High Resolution Multispectral Images Research Articles

Atlas of urban scaling laws

Accurate estimates of the urban fractal dimension D f are obtained by implementing the detrended moving average algorithm on high-resolution multi-spectral satellite images from the WorldView2 (WV2) database covering the largest European cities. Fractal dimension D f varies between 1.65 and 1.90 with high values for highly urbanised urban sectors and low ones for suburban and peripheral ones. Based on recently proposed models, the values of the fractal dimension D f are checked against the exponents β s and β i of the scaling law Y ∼ N β , respectively for socio-economic and infrastructural variables Y, with N the population size. The exponents β s and β i are traditionally derived as if cities were zero-dimensional objects, with the relevant feature Y related to a single homogeneous population value N, thus neglecting the microscopic heterogeneity of the urban structure. Our findings go beyond this limit. High sensitive and repeatable satellite records yield robust local estimates of the urban scaling exponents. Furthermore, the work discusses how to discriminate among different scaling theories, shedding light on the debated issue of scaling phenomena contradictory perspectives and pave paths to a more systematic adoption of the complex system science methods to urban landscape analysis.

FT-ICR Mass Spectrometry Imaging at Extreme Mass Resolving Power Using a Dynamically Harmonized ICR Cell with 1ω or 2ω Detection.

MALDI mass spectrometry imaging (MALDI MSI) is a powerful analytical method for achieving 2D localization of compounds from thin sections of typically but not exclusively biological samples. The dynamically harmonized ICR cell (ParaCell) was recently introduced to achieve extreme spectral resolution capable of providing the isotopic fine structure of ions detected in complex samples. The latest improvement in the ICR technology also includes 2ω detection, which significantly reduces the transient time while preserving the nominal mass resolving power of the ICR cell. High-resolution MS images acquired on FT-ICR instruments equipped with 7T and 9.4T superconducting magnets and the dynamically harmonized ICR cell operating at suboptimal parameters suffered severely from the pixel-to-pixel shifting of m/z peaks due to space-charge effects. The resulting profile average mass spectra have depreciated mass measurement accuracy and mass resolving power under the instrument specifications that affect the confidence level of the identified ions. Here, we propose an analytical workflow based on the monitoring of the total ion current to restrain the pixel-to-pixel m/z shift. Adjustment of the laser parameters is proposed to maintain high spectral resolution and mass accuracy measurement within the instrument specifications during MSI analyses. The optimized method has been successfully employed in replicates to perform high-quality MALDI MS images at resolving power (FWHM) above 1,000,000 in the lipid mass range across the whole image for superconducting magnets of 7T and 9.4T using 1 and 2ω detection. Our data also compare favorably with MALDI MSI experiments performed on higher-magnetic-field superconducting magnets, including the 21T MALDI FT-ICR prototype instrument of the NHMFL group at Tallahassee, Florida.

GRF: guided residual fusion for pansharpening

ABSTRACT Fusion of a multispectral (MS) image and a panchromatic (PAN) image is known as the pansharpening, which is a very important preprocessing step in remote sensing. Component substitution (CS)-based methods such as intensity-hue-saturation (IHS) may be the most widely used pansharpening methods and have been adopted in most of the professional software. However, the CS-based methods generally suffer from severe spectral distortions. In this paper, we propose a post-processing technique called guided residual fusion (GRF) to compensate the results of CS-based methods, which inherits high spatial fidelity from generalized IHS (GIHS, take it as an example) and greatly alleviates the spectral distortions. The main idea of our method is to estimate a residual map between the result of GIHS and the ideal high-resolution multispectral (HRMS) image. Specifically, we first obtain an approximate HRMS by inserting the pixels of the MS image according to the spatial resolution ratio between the PAN and MS. Most of the pixels in this HRMS are unknowns. We can get a masked residual map by subtracting the known values of HRMS and their corresponding counterparts of the result of GIHS. Then, we use an interpolation filter and an edge-preserving image smoothing filter to efficiently fill missing pixel values in the masked residual map, obtaining an accurate residual map. Extensive experiments on six real datasets (including datasets acquired by the Geoeye1, IKONOS, Landsat-8, QuickBird, SPOT6, and Worldview-3 sensors) demonstrate that our method outperforms the compared state-of-the-art methods, that is, the performance of our method ranks the best among all nine methods on three datasets (Geoeye1, IKONOS, and SPOT6), the second on two datasets (QuickBird and Worldview-3), and the third on the Landsat-8 dataset. The source code will be made public in https://ljy-rs.github.io/web.

DsTer: A dense spectral transformer for remote sensing spectral super-resolution

To obtain high-resolution hyperspectral data, spectral super-resolution is a popular computational imaging technique directly from high-resolution multispectral images. Besides sparse recovery, deep learning-based methods perform well in the past years for their powerful nonlinear mapping from multispectral to hyperspectral domains. However, convolutions in deep learning only focus on local information and have been blamed for the neglect of long-range relationships. Nowadays, transformer has been attracting great interest for its global attention to long-range interaction. In this study, we propose a dense spectral transformer with ResNet to achieve spectral super-resolution for multispectral remote sensing images. Combining transformer with ResNet meets the need for 3D data handling to remote sensing images as well as learning long-range relationships. Dense connection helps model exploit features from multi-level transformers. Moreover, spectral recovery results on natural data and three remote sensing data sets all prove the advantage of the proposed model. Furthermore, we also carry out classification experiments on real data to verify the dependability of the reconstructed spectra.

A Competent Convolutional Sparse Representation Model for Pan-Sharpening of Multi-Spectral Images

Two types of images are produced by Earth observation satellites, each having complementing spatial andspectral characteristics. Pan-sharpening (PS) is based on remote sensing and image fusion approach thatproduces a high spatial resolution multi-spectral image by merging spectral information from a low spatialresolution multispectral (MS) image with intrinsic spatial details from a high spatial resolution panchromatic(PAN) image. Traditional pan-sharpening methods continue to seek for a fused image that contains thenecessary spatial and spectral information. This work proposes a pan-sharpening method based on a recentinvention, convolutional sparse representation (CSR). Geometric structural characteristics are extracted fromthe PAN image using a CSR-based filtering procedure. The challenge of learning filters, convolutional basispursuit denoising (CBPDN), is handled using a modified dictionary learning method based on the concept ofAlternating Direction Method of Multipliers (ADMM). The retrieved details are put into MS bands usingapplicable weighting coefficients. Because the proposed fusion model avoids the standard patch-basedmethod, spatial and structural features are preserved while spectral quality is maintained. The spectraldistortion index SAM and the spatial measure ERGAS improve by 4.4 and 6.2 percent, respectively, whencompared to SR-based techniques. The computational complexity is reduced by 200 seconds when compared
 to the most recent SR-based fusion technique. The proposed method's efficacy is demonstrated by reduced-scale and full-scale experimental findings utilising the QuickBird and GeoEye-1 datasets.

HyperNet: A deep network for hyperspectral, multispectral, and panchromatic image fusion

Traditional approaches mainly fuse a hyperspectral image (HSI) with a high-resolution multispectral image (MSI) to improve the spatial resolution of the HSI. However, such improvement in the spatial resolution of HSIs is still limited because the spatial resolution of MSIs remains low. To further improve the spatial resolution of HSIs, we propose HyperNet, a deep network for the fusion of HSI, MSI, and panchromatic image (PAN), which effectively injects the spatial details of an MSI and a PAN into an HSI while preserving the spectral information of the HSI. Thus, we design HyperNet on the basis of a uniform fusion strategy to solve the problem of complex fusion of three types of sources (i.e., HSI, MSI, and PAN). In particular, the spatial details of the MSI and the PAN are extracted by multiple specially designed multiscale-attention-enhance blocks in which multi-scale convolution is used to adaptively extract features from different reception fields, and two attention mechanisms are adopted to enhance the representation capability of features along the spectral and spatial dimensions, respectively. Through the capability of feature reuse and interaction in a specially designed dense-detail-insertion block, the previously extracted features are subsequently injected into the HSI according to the unidirectional feature propagation among the layers of dense connection. Finally, we construct an efficient loss function by integrating the multi-scale structural similarity index with the L1 norm, which drives HyperNet to generate high-quality results with a good balance between spatial and spectral qualities. Extensive experiments on simulated and real data sets qualitatively and quantitatively demonstrate the superiority of HyperNet over other state-of-the-art methods.

Multiscale Spatial–Spectral Interaction Transformer for Pan-Sharpening

Pan-sharpening methods based on deep neural network (DNN) have produced state-of-the-art fusion performance. However, DNN-based methods mainly focus on the modeling of the local properties in low spatial resolution multispectral (LR MS) and panchromatic (PAN) images by convolution neural networks. The global dependencies in the images are ignored. To capture the local and global properties of the images concurrently, we propose a multiscale spatial–spectral interaction transformer (MSIT) for pan-sharpening. Specifically, we construct the multiscale sub-networks containing convolution–transformer encoder to extract the local and global features at different scales from LR MS and PAN images, respectively. Then, a spatial–spectral interaction attention module (SIAM) is designed to merge the features at each scale. In SIAM, the interaction attention is used to decouple the spatial and spectral information efficiently for the enhancement of complementarity and the reduction of redundancy in the extracted features. The features from different scales are further integrated into a multiscale reconstruction module (MRM) to generate the desired high spatial resolution multispectral image, in which the spatial and spectral information is fused scale by scale. The experiments on reduced- and full-scale datasets demonstrate that the proposed MSIT can produce better results in terms of visual and numerical analysis when compared with state-of-the-art methods.

Sugarcane Nitrogen Concentration and Irrigation Level Prediction Based on UAV Multispectral Imagery.

Sugarcane is the main industrial crop for sugar production, and its growth status is closely related to fertilizer, water, and light input. Unmanned aerial vehicle (UAV)-based multispectral imagery is widely used for high-throughput phenotyping, since it can rapidly predict crop vigor at field scale. This study focused on the potential of drone multispectral images in predicting canopy nitrogen concentration (CNC) and irrigation levels for sugarcane. An experiment was carried out in a sugarcane field with three irrigation levels and five fertilizer levels. Multispectral images at an altitude of 40 m were acquired during the elongating stage. Partial least square (PLS), backpropagation neural network (BPNN), and extreme learning machine (ELM) were adopted to establish CNC prediction models based on various combinations of band reflectance and vegetation indices. The simple ratio pigment index (SRPI), normalized pigment chlorophyll index (NPCI), and normalized green-blue difference index (NGBDI) were selected as model inputs due to their higher grey relational degree with the CNC and lower correlation between one another. The PLS model based on the five-band reflectance and the three vegetation indices achieved the best accuracy (Rv = 0.79, RMSEv = 0.11). Support vector machine (SVM) and BPNN were then used to classify the irrigation levels based on five spectral features which had high correlations with irrigation levels. SVM reached a higher accuracy of 80.6%. The results of this study demonstrated that high resolution multispectral images could provide effective information for CNC prediction and water irrigation level recognition for sugarcane crop.

A model-guided deep convolutional sparse coding network for hyperspectral and multispectral image fusion

ABSTRACT Although high-resolution hyperspectral images are of great significance in various applications, the manner in which to acquire such images remains a fundamental problem due to sensors’ physical limitations. Fusing high-spatial resolution multispectral images (HR-MSIs) with low-spatial resolution hyperspectral images (LR-HSIs) has become a common approach to increase the spatial resolution of hyperspectral images (HSIs). Various fusion methods have been developed to obtain images with high resolution in both the spatial and spectral domains; the main drawbacks of these methods are time consumption and spectral distortion. With the intention of developing a fast yet accurate image fusion algorithm, this paper is attempted to incorporate the benefits of deep learning and convolutional sparse coding to create a new fusion method. First, the inputs are initialized by solving the Sylvester equation from an up-sampled LR-HSI and an HR-MSI. Second, the learned iterative shrinkage thresholding algorithm is extended to its convolutional version as the central part of the model and recursively learn the sparse priors. Finally, a fused image is achieved by adding the residual to the initialized image. The proposed method is conducted on the widely used datasets (Harvard and iCVL). The experimental results are demonstrated that the proposed approach outperforms several state-of-the-art approaches in terms of quantitative and qualitative perspectives.

A Two-Stage Pansharpening Method for the Fusion of Remote-Sensing Images

The pansharpening (PS) of remote-sensing images aims to fuse a high-resolution panchromatic image with several low-resolution multispectral images for obtaining a high-resolution multispectral image. In this work, a two-stage PS model is proposed by integrating the ideas of component replacement and the variational method. The global sparse gradient of the panchromatic image is extracted by variational method, and the weight function is constructed by combining the gradient of multispectral image in which the global sparse gradient can provide more robust gradient information. Furthermore, we refine the results in order to reduce spatial and spectral distortions. Experimental results show that our method had high generalization ability for QuickBird, Gaofen-1, and WorldView-4 satellite data. Experimental results evaluated by seven metrics demonstrate that the proposed two-stage method enhanced spatial details subjective visual effects better than other state-of-the-art methods do. At the same time, in the process of quantitative evaluation, the method in this paper had high improvement compared with that other methods, and some of them can reach a maximal improvement of 60%.

Stepwise Fusion of Hyperspectral, Multispectral and Panchromatic Images with Spectral Grouping Strategy: A Comparative Study Using GF5 and GF1 Images

Since hyperspectral satellite images (HSIs) usually hold low spatial resolution, improving the spatial resolution of hyperspectral imaging (HSI) is an effective solution to explore its potential for remote sensing applications, such as land cover mapping over urban and coastal areas. The fusion of HSIs with high spatial resolution multispectral images (MSIs) and panchromatic (PAN) images could be a solution. To address the challenging work of fusing HSIs, MSIs and PAN images, a novel easy-to-implement stepwise fusion approach was proposed in this study. The fusion of HSIs and MSIs was decomposed into a set of simple image fusion tasks through spectral grouping strategy. HSI, MSI and PAN images were fused step by step using existing image fusion algorithms. According to different fusion order, two strategies ((HSI+MSI)+PAN and HSI+(MSI+PAN)) were proposed. Using simulated and real Gaofen-5 (GF-5) HSI, MSI and PAN images from the Gaofen-1 (GF-1) PMS sensor as experimental data, we compared the proposed stepwise fusion strategies with the traditional fusion strategy (HSI+PAN), and compared the performances of six fusion algorithms under three fusion strategies. We comprehensively evaluated the fused results through three aspects: spectral fidelity, spatial fidelity and computation efficiency evaluation. The results showed that (1) the spectral fidelity of the fused images obtained by stepwise fusion strategies was better than that of the traditional strategy; (2) the proposed stepwise strategies performed better or comparable spatial fidelity than traditional strategy; (3) the stepwise strategy did not significantly increase the time complexity compared to the traditional strategy; and (4) we also provide suggestions for selecting image fusion algorithms using the proposed strategy. The study provided us with a reference for the selection of fusion strategies and algorithms in different application scenarios, and also provided an easy-to-implement solution and useful references for fusing HSI, MSI and PAN images.

Identifying Urban Wetlands through Remote Sensing Scene Classification Using Deep Learning: A Case Study of Shenzhen, China

Urban wetlands provide cities with unique and valuable ecosystem services but are under great degradation pressure. Correctly identifying urban wetlands from remote sensing images is fundamental for developing appropriate management and protection plans. To overcome the semantic limitations of traditional pixel-level urban wetland classification techniques, we proposed an urban wetland identification framework based on an advanced scene-level classification scheme. First, the Sentinel-2 high-resolution multispectral image of Shenzhen was segmented into 320 m × 320 m square patches to generate sample datasets for classification. Next, twelve typical convolutional neural network (CNN) models were transformed for the comparison experiments. Finally, the model with the best performance was used to classify the wetland scenes in Shenzhen, and pattern and composition analyses were also implemented in the classification results. We found that the DenseNet121 model performed best in classifying urban wetland scenes, with overall accuracy (OA) and kappa values reaching 0.89 and 0.86, respectively. The analysis results revealed that the wetland scene in Shenzhen is generally balanced in the east–west direction. Among the wetland scenes, coastal open waters accounted for a relatively high proportion and showed an obvious southward pattern. The remaining swamp, marsh, tidal flat, and pond areas were scattered, accounting for only 4.64% of the total area of Shenzhen. For scattered and dynamic urban wetlands, we are the first to achieve scene-level classification with satisfactory results, thus providing a clearer and easier-to-understand reference for management and protection, which is of great significance for promoting harmony between humanity and ecosystems in cities.

Reconstruction of high spectral resolution multispectral image using dictionary-based learning and sparse coding

With the rising development of imaging spectroscopy, there is an increase in demand for images with high spectral, spatial, and temporal resolution. This study proposes a reconstruction from low spectral resolution multispectral imagery to higher spectral resolution hyperspectral imagery to overcome the problem of low spatial resolution and less availability of multi-temporal hyperspectral images. This study proposes a novel approach to dictionary learning based on nonlinear unmixing for dense mixtures and linear unmixing for sparse mixtures. Additionally, least square-based sparse coding has been applied in this framework to reconstruct high-resolution spectra from low-resolution spectra. The proposed method has been implemented on standard as well as real datasets and the performance of reconstruction has been validated through parametric techniques. The proposed method has been compared with states of the art methods like Joint Sparse & Low-Ranking Method and Derivative based Learning Method and it is observed that the proposed method outperforms the other methods. Nonlinear Spectral Unmixing has been performed and classification accuracy has been assessed on regenerated image data and observed to produce satisfactory output. In this study, the regenerated high spectral resolution image has been used for species-level classification and mapping of pure and mixed patches of mangrove species and the output compared with the Hyperion hyperspectral dataset of Henry Island, Sunderbans, India. Similarly, the proposed algorithm has also been applied on the Indian Pines, Salinas, Cuprite and Jasper Ridge datasets and the regenerated high spectral resolution images have been used for subpixel level target detection and classification.

A Simple, Fully Automated Shoreline Detection Algorithm for High-Resolution Multi-Spectral Imagery

This paper develops and validates a new fully automated procedure for shoreline delineation from high-resolution multispectral satellite images. The model is based on a new water–land index, the Direct Difference Water Index (DDWI). A new technique based on the buffer overlay method is also presented to determine the shoreline changes from different satellite images and obtain a time series for the shoreline changes. The shoreline detection model was applied to imagery from multiple satellites and validated to have sub-pixel accuracy using beach survey data that were collected from the Lake Michigan (USA) shoreline using a novel backpack-based LiDAR system. The model was also applied to 132 satellite images of a Lake Michigan beach over a three-year period and detected the shoreline accurately, with a >99% success rate. The model out-performed other existing shoreline detection algorithms based on different water indices and clustering techniques. The resolution shoreline position timeseries is the first satellite image-extracted dataset of its kind in terms of its high spatial and temporal resolution, and paves the road to obtaining other high-temporal-resolution datasets to refine models of beaches worldwide.

Radiative Transfer Image Simulation Using L-System Modeled Strawberry Canopies

The image-based modeling and simulation of plant growth have numerous and diverse applications. In this study, we used image-based and manual field measurements to develop and validate a methodology to simulate strawberry (Fragaria × ananassa Duch.) plant canopies throughout the Florida strawberry growing season. The simulated plants were used to create a synthetic image using radiative transfer modeling. Observed canopy properties were incorporated into an L-system simulator, and a series of strawberry canopies corresponding to specific weekly observation dates were created. The simulated canopies were compared visually with actual plant images and quantitatively with in-situ leaf area throughout the strawberry season. A simple regression model with L-system-derived and in-situ total leaf areas had an Adj R2 value of 0.78. The L-system simulated canopies were used to derive information needed for image simulation, such as leaf area and leaf angle distribution. Spectral and plant canopy information were used to create synthetic high spatial resolution multispectral images using the Discrete Anisotropic Radiative Transfer (DART) software. Vegetation spectral indices were extracted from the simulated image and used to develop multiple regression models of in-situ biophysical parameters (leaf area and dry biomass), achieving Adj R2 values of 0.63 and 0.50, respectively. The Normalized Difference Vegetation Index (NDVI) and the Red Edge Simple Ratio (SRre) vegetation indices, which utilize the red, red edge, and near infrared bands of the spectrum, were identified as statistically significant variables (p < 0.10). This study showed that both geometric (canopy seize metrics) and spectral variables were successful in modeling in-situ biomass and leaf area. Combining the geometric and spectral variables, however, only slightly improved the prediction model. These results show the feasibility of simulating strawberry canopies and images with inherent geometrical, topological, and spectral properties of real strawberry plants. The simulated canopies and images can be used in applications beyond creating realistic computer graphics for quantitative applications requiring the depiction of vegetation biological processes, such as stress modeling and remote sensing mission planning.

Evaluation of Multi-Source High-Resolution Remote Sensing Image Fusion in Aquaculture Areas

Image fusion of satellite sensors can generate a high-resolution multi-spectral image from inputs of a high spatial resolution panchromatic image and a low spatial resolution multi-spectral image for feature extraction and target recognition, such as enclosure seines and floating rafts. However, there is currently no clear and definite method of image fusion for different aquaculture areas distribution extraction from high-resolution satellite images. This study uses three types of high-resolution remote sensing images, GF-1 (Gaofen-1), GF-2 (Gaofen-2), and WV-2 (WorldView-2), covering the raft and enclosure seines aquacultures in the Xiangshan Bay, China, to evaluate panchromatic and multispectral image fusion techniques to determine which is the best. This study applied PCA (principal component analysis), GS (Gram-Schmidt), and NNDiffuse (nearest neighbor diffusion) algorithms to panchromatic and multispectral images fusion of GF-1, GF-2, and WV-2. Two quantitative methods are used to evaluate the fusion effect. The first used seven statistical parameters, including gray mean value, standard deviation, information entropy, average gradient, correlation coefficient, deviation index, and spectral distortion. The second is the CQmax index. Comparing the evaluation results by these seven common statistical indicators with the results of the image fusion evaluation by index CQmax, the results prove that the CQmax index can be applied to the evaluation of image fusion effects in different aquaculture areas. For the floating raft cultured area, the conclusion is consentaneous; NNDiffuse was also optimal for GF-1 and GF-2 data, and PCA was optimal for WV-2 data. For the enclosure seines culture area, the conclusion of quantitative evaluations is not consistent and it shows that there is no definite good method that can be applied to all areas; therefore, careful evaluation and selection of the best applicable image fusion method are required according to the study area and sensor images.

Remote Sensing Pansharpening by Full-Depth Feature Fusion

Pansharpening is an important yet challenging remote sensing image processing task, which aims to reconstruct a high-resolution (HR) multispectral (MS) image by fusing a HR panchromatic (PAN) image and a low-resolution (LR) MS image. Though deep learning (DL)-based pansharpening methods have achieved encouraging performance, they are infeasible to fully utilize the deep semantic features and shallow contextual features in the process of feature fusion for a HR-PAN image and LR-MS image. In this paper, we propose an efficient full-depth feature fusion network (FDFNet) for remote sensing pansharpening. Specifically, we design three distinctive branches called PAN-branch, MS-branch, and fusion-branch, respectively. The features extracted from the PAN and MS branches will be progressively injected into the fusion branch at every different depth to make the information fusion more broad and comprehensive. With this structure, the low-level contextual features and high-level semantic features can be characterized and integrated adequately. Extensive experiments on reduced- and full-resolution datasets acquired from WorldView-3, QuickBird, and GaoFen-2 sensors demonstrate that the proposed FDFNet only with less than 100,000 parameters performs better than other detail injection-based proposals and several state-of-the-art approaches, both visually and quantitatively.

A generative adversarial network with joint multistream architecture and spectral compensation for pansharpening

Convolutional neural networks (CNNs) and variational models for pansharpening have obtained a compelling performance gain over the state of the art. Inspired by these models, we propose MSCGAN, a generative adversarial network (GAN) with joint multistream architecture and spectral compensation for pansharpening that uses a variational model to incorporate domain-specific knowledge and in particular, introduces a spectral compensation block. First, we extract the structural information of the panchromatic (PAN) image and input it into the generator together with the upsampled multispectral (MS) image. Then, we design a multistream pansharpening CNN architecture suite for domain-specific knowledge. Second, to boost the quality of the pansharpened images, we put the MS image in the generator and design a spectral compensation block. Then, we introduce the concept of the energy function of the variational model and add corresponding spectral constraints and spatial structure constraints to the objective function to achieve a compromise between spectral information fidelity and spatial information fidelity. Finally, the discriminator also introduces spatial structure information to help the generator generate the desired high-resolution multispectral image. Experiments on the images captured by the Quickbird and WorldView-2 satellites show that the our proposed MSCGAN can make use of PAN and LRMS images adequately to attain very encouraging results, obtaining large gains over the state of the art both visually and as measured by quality metrics.

Fast and High-Quality Blind Multi-Spectral Image Pansharpening

Blind pansharpening addresses the problem of generating a high spatial-resolution multi-spectral (HRMS) image given a low spatial-resolution multi-spectral (LRMS) image with the guidance of its associated spatially misaligned high spatial-resolution panchromatic (PAN) image without parametric side information. In this paper, we propose a fast approach to blind pansharpening and achieve state-of-the-art image reconstruction quality. Typical blind pansharpening algorithms are often computationally intensive since the blur kernel and the target HRMS image are often computed using iterative solvers and in an alternating fashion. To achieve fast blind pansharpening, we decouple the solution of the blur kernel and of the HRMS image. First, we estimate the blur kernel by computing the kernel coefficients with minimum total generalized variation that blur a downsampled version of the PAN image to approximate a linear combination of the LRMS image channels. Then, we estimate each channel of the HRMS image using local Laplacian prior to regularize the relationship between each HRMS channel and the PAN image. Solving the HRMS image is accelerated by both parallelizing across the channels and by fast numerical algorithms for each channel. Due to the fast scheme and the powerful priors we used on the blur kernel coefficients (total generalized variation) and on the cross-channel relationship (local Laplacian prior), numerical experiments demonstrate that our algorithm outperforms state-of-the-art model-based counterparts in terms of both computational time and reconstruction quality of the HRMS images.

A New Context-Aware Details Injection Fidelity With Adaptive Coefficients Estimation for Variational Pansharpening

Pansharpening is related to the fusion of a low spatial resolution multispectral (MS) image retaining an abundant spectral content and a high spatial resolution panchromatic (PAN) image to obtain a product with both the abundant spectral content of the former and the high spatial resolution of the latter. Many previous studies are only focused on the global or local relationship between the PAN image and the corresponding high-resolution multispectral (HRMS) image. However, we found that the relationship between PAN and HRMS images in the gradient domain can be better explored through the image context. In this article, we propose context-aware details injection fidelity (CDIF) with adaptive coefficients estimation, which can fully explore the complicated relationship between the PAN image and the HRMS image in the gradient domain. More specifically, we apply a clustering method to divide the pixels of an image into different context-based regions. Afterward, the adaptive coefficients are estimated by using a regression-based method for each region. The CDIF is effective in extracting the main features from the two inputs to be fused. In addition, we integrate the CDIF with a conventional fidelity term and a total variation regularization to formulate a novel variational pansharpening model that is solved by designing an algorithm based on the alternating direction method of multiplier (ADMM) framework. Qualitative and quantitative assessments on different datasets support the effectiveness and robustness of the proposed method. The code is available at <uri>https://github.com/liangjiandeng/CDIF</uri>.